openstack/sunbeam-charms
Code Issues Proposed changes

first cut

Browse Source
This commit is contained in:
Guillaume Boutry
2023-10-16 17:06:56 +02:00
commit b51b266d93
26 changed files with 2857 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
charms/openstack-exporter-k8s/.gitignore vendored Normal file
View File

@@ -0,0 +1,11 @@
venv/
build/
*.charm
.tox/
.coverage
__pycache__/
*.py[cod]
.idea
.vscode/
*.swp
.stestr/

5
charms/openstack-exporter-k8s/.gitreview Normal file
View File

@@ -0,0 +1,5 @@
[gerrit]
host=review.opendev.org
port=29418
project=openstack/charm-openstack-exporter-k8s.git
defaultbranch=main

3
charms/openstack-exporter-k8s/.stestr.conf Normal file
View File

@@ -0,0 +1,3 @@
[DEFAULT]
test_path=./tests/unit
top_dir=./tests

11
charms/openstack-exporter-k8s/.zuul.yaml Normal file
View File

@@ -0,0 +1,11 @@
- project:
templates:
- openstack-python3-charm-jobs
- openstack-cover-jobs
- microk8s-func-test
vars:
charm_build_name: openstack-exporter-k8s
juju_channel: 3.2/stable
juju_classic_mode: false
microk8s_channel: 1.26-strict/stable
microk8s_classic_mode: false

34
charms/openstack-exporter-k8s/CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,34 @@
# Contributing
To make contributions to this charm, you'll need a working [development setup](https://juju.is/docs/sdk/dev-setup).
You can create an environment for development with `tox`:
```shell
tox devenv -e integration
source venv/bin/activate
```
## Testing
This project uses `tox` for managing test environments. There are some pre-configured environments
that can be used for linting and formatting code when you're preparing contributions to the charm:
```shell
tox run -e format # update your code according to linting rules
tox run -e lint # code style
tox run -e static # static type checking
tox run -e unit # unit tests
tox run -e integration # integration tests
tox # runs 'format', 'lint', 'static', and 'unit' environments
```
## Build the charm
Build the charm in this git repository using:
```shell
charmcraft pack
```
<!-- You may want to include any contribution/style guidelines in this document>

202
charms/openstack-exporter-k8s/LICENSE Normal file
View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2023 Guillaume
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

26
charms/openstack-exporter-k8s/README.md Normal file
View File

@@ -0,0 +1,26 @@
<!--
Avoid using this README file for information that is maintained or published elsewhere, e.g.:
* metadata.yaml > published on Charmhub
* documentation > published on (or linked to from) Charmhub
* detailed contribution guide > documentation or CONTRIBUTING.md
Use links instead.
-->
# openstack-exporter-k8s
Charmhub package name: operator-template
More information: https://charmhub.io/openstack-exporter-k8s
Describe your charm in one or two sentences.
## Other resources
<!-- If your charm is documented somewhere else other than Charmhub, provide a link separately. -->
- [Read more](https://example.com)
- [Contributing](CONTRIBUTING.md) <!-- or link to other contribution documentation -->
- See the [Juju SDK documentation](https://juju.is/docs/sdk) for more information about developing and improving charms.

2
charms/openstack-exporter-k8s/actions.yaml Normal file
View File

@@ -0,0 +1,2 @@
# NOTE: no actions yet!
{ }

37
charms/openstack-exporter-k8s/charmcraft.yaml Normal file
View File

@@ -0,0 +1,37 @@
type: "charm"
bases:
- build-on:
- name: "ubuntu"
channel: "22.04"
run-on:
- name: "ubuntu"
channel: "22.04"
parts:
update-certificates:
plugin: nil
override-build: |
apt update
apt install -y ca-certificates
update-ca-certificates
dashboards:
plugin: dump
source: https://github.com/openstack-exporter/grafana-dashboards
source-type: git
source-depth: 1
charm:
after: [update-certificates, dashboards]
build-packages:
- git
- libffi-dev
- libssl-dev
- rustc
- cargo
- pkg-config
charm-binary-python-packages:
- cryptography
- jsonschema
- pydantic<2.0
- jinja2
- git+https://opendev.org/openstack/charm-ops-sunbeam#egg=ops_sunbeam

27
charms/openstack-exporter-k8s/config.yaml Normal file
View File

@@ -0,0 +1,27 @@
options:
debug:
default: False
description: Enable debug logging.
type: boolean
os-admin-hostname:
default: glance.juju
description: |
The hostname or address of the admin endpoints that should be advertised
in the glance image provider.
type: string
os-internal-hostname:
default: glance.juju
description: |
The hostname or address of the internal endpoints that should be advertised
in the glance image provider.
type: string
os-public-hostname:
default: glance.juju
description: |
The hostname or address of the internal endpoints that should be advertised
in the glance image provider.
type: string
region:
default: RegionOne
description: Space delimited list of OpenStack regions
type: string

5
charms/openstack-exporter-k8s/fetch-libs.sh Executable file
View File

@@ -0,0 +1,5 @@
#!/bin/bash
echo "INFO: Fetching libs from charmhub."
charmcraft fetch-lib charms.keystone_k8s.v0.identity_resource
charmcraft fetch-lib charms.tls_certificates_interface.v1.tls_certificates

392
charms/openstack-exporter-k8s/lib/charms/keystone_k8s/v0/identity_resource.py Normal file
View File

@@ -0,0 +1,392 @@
"""IdentityResourceProvides and Requires module.
This library contains the Requires and Provides classes for handling
the identity_ops interface.
Import `IdentityResourceRequires` in your charm, with the charm object and the
relation name:
- self
- "identity_ops"
Also provide additional parameters to the charm object:
- request
Three events are also available to respond to:
- provider_ready
- provider_goneaway
- response_avaialable
A basic example showing the usage of this relation follows:
```
from charms.keystone_k8s.v0.identity_resource import IdentityResourceRequires
class IdentityResourceClientCharm(CharmBase):
def __init__(self, *args):
super().__init__(*args)
# IdentityResource Requires
self.identity_resource = IdentityResourceRequires(
self, "identity_ops",
)
self.framework.observe(
self.identity_resource.on.provider_ready, self._on_identity_resource_ready)
self.framework.observe(
self.identity_resource.on.provider_goneaway, self._on_identity_resource_goneaway)
self.framework.observe(
self.identity_resource.on.response_available, self._on_identity_resource_response)
def _on_identity_resource_ready(self, event):
'''React to the IdentityResource provider_ready event.
This event happens when n IdentityResource relation is added to the
model. Ready to send any ops to keystone.
'''
# Ready to send any ops.
pass
def _on_identity_resource_response(self, event):
'''React to the IdentityResource response_available event.
The IdentityResource interface will provide the response for the ops sent.
'''
# Read the response for the ops sent.
pass
def _on_identity_resource_goneaway(self, event):
'''React to the IdentityResource goneaway event.
This event happens when an IdentityResource relation is removed.
'''
# IdentityResource Relation has goneaway. No ops can be sent.
pass
```
A sample ops request can be of format
{
"id": <request id>
"tag": <string to identify request>
"ops": [
{
"name": <op name>,
"params": {
<param 1>: <value 1>,
<param 2>: <value 2>
}
}
]
}
For any sensitive data in the ops params, the charm can create secrets and pass
secret id instead of sensitive data as part of ops request. The charm should
ensure to grant secret access to provider charm i.e., keystone over relation.
The secret content should hold the sensitive data with same name as param name.
"""
import json
import logging
from typing import (
Optional,
)
from ops.charm import (
CharmBase,
RelationBrokenEvent,
RelationChangedEvent,
RelationEvent,
RelationJoinedEvent,
)
from ops.framework import (
EventBase,
EventSource,
Object,
ObjectEvents,
StoredState,
)
from ops.model import (
Relation,
)
logger = logging.getLogger(__name__)
# The unique Charmhub library identifier, never change it
LIBID = "b419d4d8249e423487daafc3665ed06f"
# Increment this major API version when introducing breaking changes
LIBAPI = 0
# Increment this PATCH version before using `charmcraft publish-lib` or reset
# to 0 if you are raising the major API version
LIBPATCH = 3
REQUEST_NOT_SENT = 1
REQUEST_SENT = 2
REQUEST_PROCESSED = 3
class IdentityOpsProviderReadyEvent(RelationEvent):
"""Has IdentityOpsProviderReady Event."""
pass
class IdentityOpsResponseEvent(RelationEvent):
"""Has IdentityOpsResponse Event."""
pass
class IdentityOpsProviderGoneAwayEvent(RelationEvent):
"""Has IdentityOpsProviderGoneAway Event."""
pass
class IdentityResourceResponseEvents(ObjectEvents):
"""Events class for `on`."""
provider_ready = EventSource(IdentityOpsProviderReadyEvent)
response_available = EventSource(IdentityOpsResponseEvent)
provider_goneaway = EventSource(IdentityOpsProviderGoneAwayEvent)
class IdentityResourceRequires(Object):
"""IdentityResourceRequires class."""
on = IdentityResourceResponseEvents()
_stored = StoredState()
def __init__(self, charm: CharmBase, relation_name: str):
super().__init__(charm, relation_name)
self.charm = charm
self.relation_name = relation_name
self._stored.set_default(provider_ready=False, requests=[])
self.framework.observe(
self.charm.on[relation_name].relation_joined,
self._on_identity_resource_relation_joined,
)
self.framework.observe(
self.charm.on[relation_name].relation_changed,
self._on_identity_resource_relation_changed,
)
self.framework.observe(
self.charm.on[relation_name].relation_broken,
self._on_identity_resource_relation_broken,
)
def _on_identity_resource_relation_joined(
self, event: RelationJoinedEvent
):
"""Handle IdentityResource joined."""
self._stored.provider_ready = True
self.on.provider_ready.emit(event.relation)
def _on_identity_resource_relation_changed(
self, event: RelationChangedEvent
):
"""Handle IdentityResource changed."""
id_ = self.response.get("id")
self.save_request_in_store(id_, None, None, REQUEST_PROCESSED)
self.on.response_available.emit(event.relation)
def _on_identity_resource_relation_broken(
self, event: RelationBrokenEvent
):
"""Handle IdentityResource broken."""
self._stored.provider_ready = False
self.on.provider_goneaway.emit(event.relation)
@property
def _identity_resource_rel(self) -> Optional[Relation]:
"""The IdentityResource relation."""
return self.framework.model.get_relation(self.relation_name)
@property
def response(self) -> dict:
"""Response object from keystone."""
response = self.get_remote_app_data("response")
if not response:
return {}
try:
return json.loads(response)
except Exception as e:
logger.debug(str(e))
return {}
def save_request_in_store(
self, id: str, tag: str, ops: list, state: int
) -> None:
"""Save request in the store."""
if id is None:
return
for request in self._stored.requests:
if request.get("id") == id:
if tag:
request["tag"] = tag
if ops:
request["ops"] = ops
request["state"] = state
return
# New request
self._stored.requests.append(
{"id": id, "tag": tag, "ops": ops, "state": state}
)
def get_request_from_store(self, id: str) -> dict:
"""Get request from the stote."""
for request in self._stored.requests:
if request.get("id") == id:
return request
return {}
def is_request_processed(self, id: str) -> bool:
"""Check if request is processed."""
for request in self._stored.requests:
if (
request.get("id") == id
and request.get("state") == REQUEST_PROCESSED
):
return True
return False
def get_remote_app_data(self, key: str) -> Optional[str]:
"""Return the value for the given key from remote app data."""
if self._identity_resource_rel:
data = self._identity_resource_rel.data[
self._identity_resource_rel.app
]
return data.get(key)
return None
def ready(self) -> bool:
"""Interface is ready or not.
Interface is considered ready if the op request is processed
and response is sent. In case of non leader unit, just consider
the interface is ready.
"""
if not self.model.unit.is_leader():
logger.debug("Not a leader unit, set the interface to ready")
return True
try:
app_data = self._identity_resource_rel.data[self.charm.app]
if "request" not in app_data:
return False
request = json.loads(app_data["request"])
request_id = request.get("id")
response_id = self.response.get("id")
if request_id == response_id:
return True
except Exception as e:
logger.debug(str(e))
return False
def request_ops(self, request: dict) -> None:
"""Request keystone ops."""
if not self.model.unit.is_leader():
logger.debug("Not a leader unit, not sending request")
return
id_ = request.get("id")
tag = request.get("tag")
ops = request.get("ops")
req = self.get_request_from_store(id_)
if req and req.get("state") == REQUEST_PROCESSED:
logger.debug("Request {id_} already processed")
return
if not self._stored.provider_ready:
self.save_request_in_store(id_, tag, ops, REQUEST_NOT_SENT)
logger.debug("Keystone not yet ready to take requests")
return
logger.debug("Requesting ops to keystone")
app_data = self._identity_resource_rel.data[self.charm.app]
app_data["request"] = json.dumps(request)
self.save_request_in_store(id_, tag, ops, REQUEST_SENT)
class IdentityOpsRequestEvent(EventBase):
"""Has IdentityOpsRequest Event."""
def __init__(self, handle, relation_id, relation_name, request):
"""Initialise event."""
super().__init__(handle)
self.relation_id = relation_id
self.relation_name = relation_name
self.request = request
def snapshot(self):
"""Snapshot the event."""
return {
"relation_id": self.relation_id,
"relation_name": self.relation_name,
"request": self.request,
}
def restore(self, snapshot):
"""Restore the event."""
super().restore(snapshot)
self.relation_id = snapshot["relation_id"]
self.relation_name = snapshot["relation_name"]
self.request = snapshot["request"]
class IdentityResourceProviderEvents(ObjectEvents):
"""Events class for `on`."""
process_op = EventSource(IdentityOpsRequestEvent)
class IdentityResourceProvides(Object):
"""IdentityResourceProvides class."""
on = IdentityResourceProviderEvents()
def __init__(self, charm: CharmBase, relation_name: str):
super().__init__(charm, relation_name)
self.charm = charm
self.relation_name = relation_name
self.framework.observe(
self.charm.on[relation_name].relation_changed,
self._on_identity_resource_relation_changed,
)
def _on_identity_resource_relation_changed(
self, event: RelationChangedEvent
):
"""Handle IdentityResource changed."""
request = event.relation.data[event.relation.app].get("request", {})
self.on.process_op.emit(
event.relation.id, event.relation.name, request
)
def set_ops_response(
self, relation_id: str, relation_name: str, ops_response: dict
) -> None:
"""Set response to ops request."""
if not self.model.unit.is_leader():
logger.debug("Not a leader unit, not sending response")
return
logger.debug("Update response from keystone")
_identity_resource_rel = self.charm.model.get_relation(
relation_name, relation_id
)
if not _identity_resource_rel:
# Relation has disappeared so skip send of data
return
app_data = _identity_resource_rel.data[self.charm.app]
app_data["response"] = json.dumps(ops_response)

1360
charms/openstack-exporter-k8s/lib/charms/tls_certificates_interface/v1/tls_certificates.py Normal file
View File

File diff suppressed because it is too large Load Diff

38
charms/openstack-exporter-k8s/metadata.yaml Normal file
View File

@@ -0,0 +1,38 @@
name: openstack-exporter-k8s
summary: OpenStack openstack-exporter service
maintainer: OpenStack Charmers <openstack-charmers@lists.ubuntu.com>
description: |
OpenStack openstack-exporter provides endpoint metrics ma boi
version: 3
bases:
- name: ubuntu
channel: 22.04/stable
assumes:
- k8s-api
- juju >= 3.1
tags:
- openstack
source: https://opendev.org/openstack/charm-openstack-exporter-k8s
issues: https://bugs.launchpad.net/charm-openstack-exporter-k8s
containers:
openstack-exporter:
resource: openstack-exporter-image
resources:
openstack-exporter-image:
type: oci-image
description: OCI image for OpenStack openstack-exporter
upstream-source: ghcr.io/canonical/openstack-exporter:2023.2
requires:
# identity-service:
# interface: keystone
identity-ops:
interface: keystone-resources
# certificates:
# interface: tls-certificates
peers:
peers:
interface: openstack-exporter-peer

10
charms/openstack-exporter-k8s/osci.yaml Normal file
View File

@@ -0,0 +1,10 @@
- project:
templates:
- charm-publish-jobs
vars:
needs_charm_build: true
charm_build_name: openstack-exporter-k8s
build_type: charmcraft
publish_charm: true
charmcraft_channel: 2.0/stable
publish_channel: 2023.1/edge

39
charms/openstack-exporter-k8s/pyproject.toml Normal file
View File

@@ -0,0 +1,39 @@
# Copyright 2023 Canonical Ltd.
# See LICENSE file for licensing details.
# Testing tools configuration
[tool.coverage.run]
branch = true
[tool.coverage.report]
show_missing = true
[tool.pytest.ini_options]
minversion = "6.0"
log_cli_level = "INFO"
# Formatting tools configuration
[tool.black]
line-length = 79
[tool.isort]
profile = "black"
multi_line_output = 3
force_grid_wrap = true
# Linting tools configuration
[tool.flake8]
max-line-length = 79
max-doc-length = 99
max-complexity = 10
exclude = [".git", "__pycache__", ".tox", "build", "dist", "*.egg_info", "venv"]
select = ["E", "W", "F", "C", "N", "R", "D", "H"]
# Ignore W503, E501 because using black creates errors with this
# Ignore D107 Missing docstring in __init__
ignore = ["W503", "E501", "D107", "E402"]
per-file-ignores = []
docstring-convention = "google"
# Check for properly formatted copyright header in each file
copyright-check = "True"
copyright-author = "Canonical Ltd."
copyright-regexp = "Copyright\\s\\d{4}([-,]\\d{4})*\\s+%(author)s"

13
charms/openstack-exporter-k8s/rename.sh Executable file
View File

@@ -0,0 +1,13 @@
#!/bin/bash
charm=$(grep "charm_build_name" osci.yaml | awk '{print $2}')
echo "renaming ${charm}_*.charm to ${charm}.charm"
echo -n "pwd: "
pwd
ls -al
echo "Removing bad downloaded charm maybe?"
if [[ -e "${charm}.charm" ]];
then
rm "${charm}.charm"
fi
echo "Renaming charm here."
mv ${charm}_*.charm ${charm}.charm

4
charms/openstack-exporter-k8s/requirements.txt Normal file
View File

@@ -0,0 +1,4 @@
ops
jinja2
git+https://github.com/openstack/charm-ops-sunbeam#egg=ops_sunbeam
pwgen

332
charms/openstack-exporter-k8s/src/charm.py Executable file
View File

@@ -0,0 +1,332 @@
#!/usr/bin/env python3
"""Openstack-exporter Operator Charm.
This charm provide Openstack-exporter services as part of an OpenStack deployment
"""
import hashlib
import json
import logging
from typing import TYPE_CHECKING, List, Optional
import ops
import pwgen
from ops.main import main
import charms.keystone_k8s.v0.identity_resource as identity_resource
import ops_sunbeam.container_handlers as sunbeam_chandlers
import ops_sunbeam.charm as sunbeam_charm
import ops_sunbeam.config_contexts as sunbeam_config_contexts
import ops_sunbeam.core as sunbeam_core
import ops_sunbeam.relation_handlers as sunbeam_rhandlers
logger = logging.getLogger(__name__)
CREDENTIALS_SECRET_PREFIX = "credentials_"
CONTAINER = "openstack-exporter"
class OSExporterConfigurationContext(sunbeam_config_contexts.ConfigContext):
"""OSExporter configuration context."""
if TYPE_CHECKING:
charm: "OSExporterOperatorCharm"
@property
def ready(self) -> bool:
"""Whether the context has all the data is needs."""
return self.charm.auth_url is not None
def context(self) -> dict:
"""OS Exporter configuration context."""
username, password = self.charm.user_credentials
return {
"domain_name": self.charm.domain,
"project_name": self.charm.project,
"username": username,
"password": password,
"auth_url": self.charm.auth_url,
}
class OSExporterPebbleHandler(sunbeam_chandlers.ServicePebbleHandler):
def get_layer(self) -> dict:
"""Pebble configuration layer for the container."""
return {
"summary": "openstack-exporter service",
"description": ("Pebble config layer for openstack-exporter"),
"services": {
self.service_name: {
"override": "replace",
"summary": "Openstack-Exporter",
"command": (
"openstack-exporter"
" --os-client-config /etc/os-exporter/clouds.yaml"
" --multi-cloud"
),
"startup": "disabled",
},
},
}
class OSExporterOperatorCharm(sunbeam_charm.OSBaseOperatorCharmK8S):
"""Charm the service."""
mandatory_relations = {
# "certificates",
"identity-ops",
}
service_name = "openstack-exporter"
@property
def container_configs(self) -> List[sunbeam_core.ContainerConfigFile]:
"""Container configuration files for the operator."""
return [
sunbeam_core.ContainerConfigFile(
"/etc/os-exporter/clouds.yaml",
"_daemon_",
"_daemon_",
),
# sunbeam_core.ContainerConfigFile(
# "/etc/ssl/ca.pem",
# "_daemon_",
# "_daemon_",
# ),
]
@property
def config_contexts(self) -> List[sunbeam_config_contexts.ConfigContext]:
"""Generate list of configuration adapters for the charm."""
_cadapters = super().config_contexts
_cadapters.append(OSExporterConfigurationContext(self, "os_exporter"))
return _cadapters
@property
def service_conf(self) -> str:
"""Service default configuration file."""
return "/etc/os-exporter/clouds.yaml"
@property
def service_user(self) -> str:
"""Service user file and directory ownership."""
return "_daemon_"
@property
def service_group(self) -> str:
"""Service group file and directory ownership."""
return "_daemon_"
@property
def default_public_ingress_port(self):
"""Ingress Port for API service."""
return 9180
@property
def os_exporter_user(self) -> str:
"""User for openstack-exporter."""
return "openstack-exporter"
@property
def domain(self):
"""Domain name for openstack-exporter."""
return "default"
@property
def project(self):
"""Project name for openstack-exporter."""
return "services"
@property
def user_credentials(self) -> tuple:
"""Credentials for domain admin user."""
credentials_id = self._get_os_exporter_credentials_secret()
credentials = self.model.get_secret(id=credentials_id)
username = credentials.get_content().get("username")
user_password = credentials.get_content().get("password")
return (username, user_password)
@property
def auth_url(self) -> Optional[str]:
"""Auth url for openstack-exporter."""
for op in self.id_ops.interface.response.get("ops"):
if op.get("name") != "list_endpoint":
continue
for endpoint in op.get("value", []):
return endpoint.get("url")
return None
def get_relation_handlers(self) -> List[sunbeam_rhandlers.RelationHandler]:
"""Relation handlers for the service."""
handlers = super().get_relation_handlers()
self.id_ops = sunbeam_rhandlers.IdentityResourceRequiresHandler(
self,
"identity-ops",
self.handle_keystone_ops,
mandatory="identity-ops" in self.mandatory_relations,
)
handlers.append(self.id_ops)
return handlers
def get_pebble_handlers(
self,
) -> List[sunbeam_chandlers.PebbleHandler]:
"""Pebble handlers for operator."""
return [
OSExporterPebbleHandler(
self,
CONTAINER,
self.service_name,
self.container_configs,
self.template_dir,
self.configure_charm,
),
]
def hash_ops(self, ops: list) -> str:
"""Return the sha1 of the requested ops."""
return hashlib.sha1(json.dumps(ops).encode()).hexdigest()
def _grant_os_exporter_credentials_secret(
self,
relation: ops.Relation,
) -> None:
"""Grant secret access to the related units."""
credentials_id = None
try:
credentials_id = self._get_os_exporter_credentials_secret()
secret = self.model.get_secret(id=credentials_id)
logger.debug(
f"Granting access to secret {credentials_id} for relation "
f"{relation.app.name} {relation.name}/{relation.id}"
)
secret.grant(relation)
except (ops.ModelError, ops.SecretNotFoundError) as e:
logger.debug(
f"Error during granting access to secret {credentials_id} for "
f"relation {relation.app.name} {relation.name}/{relation.id}: "
f"{str(e)}"
)
def _retrieve_or_set_secret(
self,
username: str,
rotate: ops.SecretRotate = ops.SecretRotate.NEVER,
add_suffix_to_username: bool = False,
) -> str:
"""Retrieve or create a secret."""
label = f"{CREDENTIALS_SECRET_PREFIX}{username}"
credentials_id = self.peers.get_app_data(label)
if credentials_id:
return credentials_id
password = str(pwgen.pwgen(12))
if add_suffix_to_username:
suffix = pwgen.pwgen(6)
username = f"{username}-{suffix}"
credentials_secret = self.model.app.add_secret(
{"username": username, "password": password},
label=label,
rotate=rotate,
)
self.peers.set_app_data(
{
label: credentials_secret.id,
}
)
return credentials_secret.id
def _get_os_exporter_credentials_secret(self) -> str:
"""Get domain admin secret."""
label = f"{CREDENTIALS_SECRET_PREFIX}{self.os_exporter_user}"
credentials_id = self.peers.get_app_data(label)
if not credentials_id:
credentials_id = self._retrieve_or_set_secret(
self.os_exporter_user,
)
return credentials_id
def _get_os_exporter_user_ops(self) -> list:
"""Generate ops request for domain setup."""
credentials_id = self._get_os_exporter_credentials_secret()
ops = [
# show domain default
{
"name": "show_domain",
"params": {"name": "default"},
},
# fetch keystone endpoint
{
"name": "list_endpoint",
"params": {"name": "keystone", "interface": "admin"},
},
# Create user openstack exporter
{
"name": "create_user",
"params": {
"name": self.os_exporter_user,
"password": credentials_id,
"domain": "default",
},
},
# check with reader system scoped permissions
]
return ops
def _handle_initial_os_exporter_user_setup_response(
self,
event: ops.RelationEvent,
) -> None:
"""Handle domain setup response from identity-ops."""
if {
op.get("return-code")
for op in self.id_ops.interface.response.get(
"ops",
[],
)
} == {0}:
logger.debug(
"Initial openstack exporter user setup commands completed,"
" running configure charm"
)
self.configure_charm(event)
def handle_keystone_ops(self, event: ops.RelationEvent) -> None:
"""Event handler for identity ops."""
if isinstance(event, identity_resource.IdentityOpsProviderReadyEvent):
self._state.identity_ops_ready = True
if not self.unit.is_leader():
return
# Send op request only by leader unit
ops = self._get_os_exporter_user_ops()
id_ = self.hash_ops(ops)
self._grant_os_exporter_credentials_secret(event.relation)
request = {
"id": id_,
"tag": "initial_openstack_exporter_user_setup",
"ops": ops,
}
logger.debug(f"Sending ops request: {request}")
self.id_ops.interface.request_ops(request)
elif isinstance(
event,
identity_resource.IdentityOpsProviderGoneAwayEvent,
):
self._state.identity_ops_ready = False
elif isinstance(event, identity_resource.IdentityOpsResponseEvent):
if not self.unit.is_leader():
return
response = self.id_ops.interface.response
logger.debug(f"Got response from keystone: {response}")
request_tag = response.get("tag")
if request_tag == "initial_openstack_exporter_user_setup":
self._handle_initial_os_exporter_user_setup_response(event)
if __name__ == "__main__":
main(OSExporterOperatorCharm)

1
charms/openstack-exporter-k8s/src/templates/ca.pem.j2 Normal file
View File

@@ -0,0 +1 @@
{{ certificates.ca_cert }}

14
charms/openstack-exporter-k8s/src/templates/clouds.yaml.j2 Normal file
View File

@@ -0,0 +1,14 @@
clouds:
default:
region_name: {{ options.region }}
identity_api_version: 3
identity_interface: admin
auth:
username: {{ os_exporter.username }}
password: {{ os_exporter.password }}
project_name: {{ os_exporter.project_name }}
project_domain_name: {{ os_exporter.domain_name }}
user_domain_name: {{ os_exporter.domain_name }}
auth_url: {{ os_exporter.auth_url }}
# cacert: /etc/ssl/ca.pem
verify: false

9
charms/openstack-exporter-k8s/test-requirements.txt Normal file
View File

@@ -0,0 +1,9 @@
# This file is managed centrally. If you find the need to modify this as a
# one-off, please don't. Intead, consult #openstack-charms and ask about
# requirements management in charms via bot-control. Thank you.
coverage
mock
flake8
stestr
ops

35
charms/openstack-exporter-k8s/tests/integration/test_charm.py Normal file
View File

@@ -0,0 +1,35 @@
#!/usr/bin/env python3
# Copyright 2023 Guillaume
# See LICENSE file for licensing details.
import asyncio
import logging
from pathlib import Path
import pytest
import yaml
from pytest_operator.plugin import OpsTest
logger = logging.getLogger(__name__)
METADATA = yaml.safe_load(Path("./metadata.yaml").read_text())
APP_NAME = METADATA["name"]
@pytest.mark.abort_on_fail
async def test_build_and_deploy(ops_test: OpsTest):
"""Build the charm-under-test and deploy it together with related charms.
Assert on the unit status before any relations/configurations take place.
"""
# Build and deploy charm from local source folder
charm = await ops_test.build_charm(".")
resources = {"httpbin-image": METADATA["resources"]["httpbin-image"]["upstream-source"]}
# Deploy the charm and wait for active/idle status
await asyncio.gather(
ops_test.model.deploy(charm, resources=resources, application_name=APP_NAME),
ops_test.model.wait_for_idle(
apps=[APP_NAME], status="active", raise_on_blocked=True, timeout=1000
),
)

18
charms/openstack-exporter-k8s/tests/tests.yaml Normal file
View File

@@ -0,0 +1,18 @@
gate_bundles:
- smoke
smoke_bundles:
- smoke
configure:
- zaza.openstack.charm_tests.keystone.setup.add_tempest_roles
tests: []
tests_options:
trust:
- smoke
ignore_hard_deploy_errors:
- smoke
tempest:
default:
smoke: True
target_deploy_status: []

68
charms/openstack-exporter-k8s/tests/unit/test_charm.py Normal file
View File

@@ -0,0 +1,68 @@
# Copyright 2023 Guillaume
# See LICENSE file for licensing details.
#
# Learn more about testing at: https://juju.is/docs/sdk/testing
import unittest
import ops
import ops.testing
from charm import OpenstackExporterK8SCharm
class TestCharm(unittest.TestCase):
def setUp(self):
self.harness = ops.testing.Harness(OpenstackExporterK8SCharm)
self.addCleanup(self.harness.cleanup)
self.harness.begin()
def test_httpbin_pebble_ready(self):
# Expected plan after Pebble ready with default config
expected_plan = {
"services": {
"httpbin": {
"override": "replace",
"summary": "httpbin",
"command": "gunicorn -b 0.0.0.0:80 httpbin:app -k gevent",
"startup": "enabled",
"environment": {"GUNICORN_CMD_ARGS": "--log-level info"},
}
},
}
# Simulate the container coming up and emission of pebble-ready event
self.harness.container_pebble_ready("httpbin")
# Get the plan now we've run PebbleReady
updated_plan = self.harness.get_container_pebble_plan("httpbin").to_dict()
# Check we've got the plan we expected
self.assertEqual(expected_plan, updated_plan)
# Check the service was started
service = self.harness.model.unit.get_container("httpbin").get_service("httpbin")
self.assertTrue(service.is_running())
# Ensure we set an ActiveStatus with no message
self.assertEqual(self.harness.model.unit.status, ops.ActiveStatus())
def test_config_changed_valid_can_connect(self):
# Ensure the simulated Pebble API is reachable
self.harness.set_can_connect("httpbin", True)
# Trigger a config-changed event with an updated value
self.harness.update_config({"log-level": "debug"})
# Get the plan now we've run PebbleReady
updated_plan = self.harness.get_container_pebble_plan("httpbin").to_dict()
updated_env = updated_plan["services"]["httpbin"]["environment"]
# Check the config change was effective
self.assertEqual(updated_env, {"GUNICORN_CMD_ARGS": "--log-level debug"})
self.assertEqual(self.harness.model.unit.status, ops.ActiveStatus())
def test_config_changed_valid_cannot_connect(self):
# Trigger a config-changed event with an updated value
self.harness.update_config({"log-level": "debug"})
# Check the charm is in WaitingStatus
self.assertIsInstance(self.harness.model.unit.status, ops.WaitingStatus)
def test_config_changed_invalid(self):
# Ensure the simulated Pebble API is reachable
self.harness.set_can_connect("httpbin", True)
# Trigger a config-changed event with an updated value
self.harness.update_config({"log-level": "foobar"})
# Check the charm is in BlockedStatus
self.assertIsInstance(self.harness.model.unit.status, ops.BlockedStatus)

161
charms/openstack-exporter-k8s/tox.ini Normal file
View File

@@ -0,0 +1,161 @@
# Operator charm (with zaza): tox.ini
[tox]
skipsdist = True
envlist = pep8,py3
sitepackages = False
skip_missing_interpreters = False
minversion = 3.18.0
[vars]
src_path = {toxinidir}/src/
tst_path = {toxinidir}/tests/
lib_path = {toxinidir}/lib/
pyproject_toml = {toxinidir}/pyproject.toml
all_path = {[vars]src_path} {[vars]tst_path}
[testenv]
basepython = python3
setenv =
PYTHONPATH = {toxinidir}:{[vars]lib_path}:{[vars]src_path}
passenv =
HOME
PYTHONPATH
install_command =
pip install {opts} {packages}
commands = stestr run --slowest {posargs}
allowlist_externals =
git
charmcraft
{toxinidir}/fetch-libs.sh
{toxinidir}/rename.sh
deps =
-r{toxinidir}/test-requirements.txt
[testenv:fmt]
description = Apply coding style standards to code
deps =
black
isort
commands =
isort {[vars]all_path} --skip-glob {[vars]lib_path} --skip {toxinidir}/.tox
black --config {[vars]pyproject_toml} {[vars]all_path} --exclude {[vars]lib_path}
[testenv:build]
basepython = python3
deps =
commands =
charmcraft -v pack
{toxinidir}/rename.sh
[testenv:fetch]
basepython = python3
deps =
commands =
{toxinidir}/fetch-libs.sh
[testenv:py3]
basepython = python3
deps =
{[testenv]deps}
-r{toxinidir}/requirements.txt
[testenv:py38]
basepython = python3.8
deps = {[testenv:py3]deps}
[testenv:py39]
basepython = python3.9
deps = {[testenv:py3]deps}
[testenv:py310]
basepython = python3.10
deps = {[testenv:py3]deps}
[testenv:cover]
basepython = python3
deps = {[testenv:py3]deps}
setenv =
{[testenv]setenv}
PYTHON=coverage run
commands =
coverage erase
stestr run --slowest {posargs}
coverage combine
coverage html -d cover
coverage xml -o cover/coverage.xml
coverage report
[testenv:pep8]
description = Alias for lint
deps = {[testenv:lint]deps}
commands = {[testenv:lint]commands}
[testenv:lint]
description = Check code against coding style standards
deps =
black
flake8<6 # Pin version until https://github.com/savoirfairelinux/flake8-copyright/issues/19 is merged
flake8-docstrings
flake8-copyright
flake8-builtins
pyproject-flake8
pep8-naming
isort
codespell
commands =
codespell {[vars]all_path}
# pflake8 wrapper supports config from pyproject.toml
pflake8 --exclude {[vars]lib_path} --config {toxinidir}/pyproject.toml {[vars]all_path}
isort --check-only --diff {[vars]all_path} --skip-glob {[vars]lib_path}
black --config {[vars]pyproject_toml} --check --diff {[vars]all_path} --exclude {[vars]lib_path}
[testenv:func-noop]
basepython = python3
deps =
git+https://github.com/openstack-charmers/zaza.git@libjuju-3.1#egg=zaza
git+https://github.com/openstack-charmers/zaza-openstack-tests.git#egg=zaza.openstack
git+https://opendev.org/openstack/tempest.git#egg=tempest
commands =
functest-run-suite --help
[testenv:func]
basepython = python3
deps = {[testenv:func-noop]deps}
commands =
functest-run-suite --keep-model
[testenv:func-smoke]
basepython = python3
deps = {[testenv:func-noop]deps}
setenv =
TEST_MODEL_SETTINGS = automatically-retry-hooks=true
TEST_MAX_RESOLVE_COUNT = 5
commands =
functest-run-suite --keep-model --smoke
[testenv:func-dev]
basepython = python3
deps = {[testenv:func-noop]deps}
commands =
functest-run-suite --keep-model --dev
[testenv:func-target]
basepython = python3
deps = {[testenv:func-noop]deps}
commands =
functest-run-suite --keep-model --bundle {posargs}
[coverage:run]
branch = True
concurrency = multiprocessing
parallel = True
source =
.
omit =
.tox/*
tests/*
src/templates/*
[flake8]
ignore=E226,W504