ai-workspace-infra/artifacts
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Migrate to readme yaml format (#7)

Browse Source 
* feat(README): migrate to readme yaml format

* Update README and helm chart
This commit is contained in:
Vadim Hleif 2018-09-21 19:50:11 +02:00 committed by Erik Osterman
parent 661437149a
commit accffcb325
9 changed files with 309 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -18,4 +18,5 @@
*.iml
.build-harness
build-harness
dist/bin/*

2
Dockerfile
View File

@ -5,7 +5,7 @@ COPY . .
RUN go get && CGO_ENABLED=0 go build -v -o "./dist/bin/prometheus-to-cloudwatch" *.go
FROM alpine:3.7
FROM alpine:3.8
RUN apk add --no-cache ca-certificates
COPY --from=builder /go/src/github.com/cloudposse/prometheus-to-cloudwatch/dist/bin/prometheus-to-cloudwatch /usr/bin/prometheus-to-cloudwatch
ENV PATH $PATH:/usr/bin

144
README.md
View File

@ -1,10 +1,41 @@
# prometheus-to-cloudwatch [![Build Status](https://travis-ci.org/cloudposse/prometheus-to-cloudwatch.svg?branch=master)](https://travis-ci.org/cloudposse/prometheus-to-cloudwatch)
<!-- This file was automatically generated by the `build-harness`. Make all changes to `README.yaml` and run `make readme` to rebuild this file. -->
[![Cloud Posse](https://cloudposse.com/logo-300x69.svg)](https://cloudposse.com)
# prometheus-to-cloudwatch [![Build Status](https://travis-ci.org/cloudposse/prometheus-to-cloudwatch.svg?branch=master)](https://travis-ci.org/cloudposse/prometheus-to-cloudwatch) [![Latest Release](https://img.shields.io/github/release/cloudposse/prometheus-to-cloudwatch.svg)](https://github.com/cloudposse/prometheus-to-cloudwatch/releases/latest) [![Slack Community](https://slack.cloudposse.com/badge.svg)](https://slack.cloudposse.com)
Utility for scraping Prometheus metrics from a Prometheus client endpoint and publishing them to CloudWatch
---
This project is part of our comprehensive ["SweetOps"](https://docs.cloudposse.com) approach towards DevOps.
It's 100% Open Source and licensed under the [APACHE2](LICENSE).
## Screenshots
![kube-state-metrics-service](images/kube-state-metrics-service.png)
*Kube state metrics*
![kube-state-metrics-to-cloudwatch](images/kube-state-metrics-to-cloudwatch.png)
*Kube state metrics to cloudwatch*
## Usage
__NOTE__: The module accepts parameters as command-line arguments or as ENV variables (or any combination of command-line arguments and ENV vars).
Command-line arguments take precedence over ENV vars
@ -32,10 +63,11 @@ which will provide IAM credentials to containers running inside a Kubernetes clu
to publish metrics to CloudWatch.
## Examples
### Build Go program
```sh
go get
@ -108,10 +140,7 @@ Find the running services
kubectl get services
```
![kube-state-metrics-service](images/kube-state-metrics-service.png)
Copy the name of the `kube-state-metrics` service (`gauche-turtle-kube-state-metrics`) into the ENV var `PROMETHEUS_SCRAPE_URL` in [values.yaml](chart/values.yaml).
Copy the name of the `kube-state-metrics` service (e.g. `gauche-turtle-kube-state-metrics`) into the ENV var `PROMETHEUS_SCRAPE_URL` in [values.yaml](chart/values.yaml).
It should look like this:
```sh
@ -131,12 +160,45 @@ helm install .
![kube-state-metrics-to-cloudwatch](images/kube-state-metrics-to-cloudwatch.png)
## Related Projects
Check out these related projects.
- [Prometheus Operator](https://github.com/cloudposse/prometheus-operator) - Prometheus Operator creates/configures/manages Prometheus clusters atop Kubernetes
- [terraform-aws-cloudwatch-logs](https://github.com/cloudposse/terraform-aws-cloudwatch-logs) - Terraform Module to Provide a CloudWatch Logs Endpoint
- [terraform-aws-ecs-web-app](https://github.com/cloudposse/terraform-aws-ecs-web-app) - Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more.
## Help
**Got a question?**
File a GitHub [issue](https://github.com/cloudposse/prometheus-to-cloudwatch/issues), send us an [email](mailto:hello@cloudposse.com) or reach out to us on [Gitter](https://gitter.im/cloudposse/).
File a GitHub [issue](https://github.com/cloudposse/prometheus-to-cloudwatch/issues), send us an [email][email] or join our [Slack Community][slack].
## Commercial Support
Work directly with our team of DevOps experts via email, slack, and video conferencing.
We provide [*commercial support*][commercial_support] for all of our [Open Source][github] projects. As a *Dedicated Support* customer, you have access to our team of subject matter experts at a fraction of the cost of a full-time engineer.
[![E-Mail](https://img.shields.io/badge/email-hello@cloudposse.com-blue.svg)](mailto:hello@cloudposse.com)
- **Questions.** We'll use a Shared Slack channel between your team and ours.
- **Troubleshooting.** We'll help you triage why things aren't working.
- **Code Reviews.** We'll review your Pull Requests and provide constructive feedback.
- **Bug Fixes.** We'll rapidly work to fix any bugs in our projects.
- **Build New Terraform Modules.** We'll develop original modules to provision infrastructure.
- **Cloud Architecture.** We'll assist with your cloud strategy and design.
- **Implementation.** We'll provide hands-on support to implement our reference architectures.
## Community Forum
Get access to our [Open Source Community Forum][slack] on Slack. It's **FREE** to join for everyone! Our "SweetOps" community is where you get to talk with others who share a similar vision for how to rollout and manage infrastructure. This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build *sweet* infrastructure.
## Contributing
@ -146,7 +208,7 @@ Please use the [issue tracker](https://github.com/cloudposse/prometheus-to-cloud
### Developing
If you are interested in being a contributor and want to get involved in developing `prometheus-to-cloudwatch`, we would love to hear from you! Shoot us an [email](mailto:hello@cloudposse.com).
If you are interested in being a contributor and want to get involved in developing this project or [help out](https://github.com/orgs/cloudposse/projects/3) with our other projects, we would love to hear from you! Shoot us an [email](mailto:hello@cloudposse.com).
In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow.
@ -154,14 +216,20 @@ In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow.
2. **Clone** the project to your own machine
3. **Commit** changes to your own branch
4. **Push** your work back up to your fork
5. Submit a **Pull request** so that we can review your changes
5. Submit a **Pull Request** so that we can review your changes
**NOTE:** Be sure to merge the latest from "upstream" before making a pull request!
**NOTE:** Be sure to merge the latest changes from "upstream" before making a pull request!
## License
## Copyright
[APACHE 2.0](LICENSE) © 2018 [Cloud Posse, LLC](https://cloudposse.com)
Copyright © 2017-2018 [Cloud Posse, LLC](https://cloudposse.com)
## License
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
See [LICENSE](LICENSE) for full details.
@ -173,7 +241,7 @@ See [LICENSE](LICENSE) for full details.
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
@ -183,31 +251,53 @@ See [LICENSE](LICENSE) for full details.
under the License.
## Trademarks
All other trademarks referenced herein are the property of their respective owners.
## About
`prometheus-to-cloudwatch` is maintained and funded by [Cloud Posse, LLC][website].
This project is maintained and funded by [Cloud Posse, LLC][website]. Like it? Please let us know at <hello@cloudposse.com>
![Cloud Posse](https://cloudposse.com/logo-300x69.png)
[![Cloud Posse](https://cloudposse.com/logo-300x69.svg)](https://cloudposse.com)
We're a [DevOps Professional Services][hire] company based in Los Angeles, CA. We love [Open Source Software](https://github.com/cloudposse/)!
Like it? Please let us know at <hello@cloudposse.com>
We offer paid support on all of our projects.
We love [Open Source Software](https://github.com/cloudposse/)!
See [our other projects][community]
or [hire us][hire] to help build your next cloud platform.
Check out [our other projects][github], [apply for a job][jobs], or [hire us][hire] to help with your cloud strategy and implementation.
[docs]: https://docs.cloudposse.com/
[website]: https://cloudposse.com/
[community]: https://github.com/cloudposse/
[github]: https://github.com/cloudposse/
[commercial_support]: https://github.com/orgs/cloudposse/projects
[jobs]: https://cloudposse.com/jobs/
[hire]: https://cloudposse.com/contact/
[slack]: https://slack.cloudposse.com/
[linkedin]: https://www.linkedin.com/company/cloudposse
[twitter]: https://twitter.com/cloudposse/
[email]: mailto:hello@cloudposse.com
### Contributors
| [![Erik Osterman][erik_img]][erik_web]<br/>[Erik Osterman][erik_web] | [![Andriy Knysh][andriy_img]][andriy_web]<br/>[Andriy Knysh][andriy_web] |
|-------------------------------------------------------|------------------------------------------------------------------|
| [![Erik Osterman][osterman_avatar]][osterman_homepage]<br/>[Erik Osterman][osterman_homepage] | [![Andriy Knysh][aknysh_avatar]][aknysh_homepage]<br/>[Andriy Knysh][aknysh_homepage] | [![Igor Rodionov][goruha_avatar]][goruha_homepage]<br/>[Igor Rodionov][goruha_homepage] | [![yufukui-m][yufukui-m_avatar]][yufukui-m_homepage]<br/>[yufukui-m][yufukui-m_homepage] |
|---|---|---|---|
[osterman_homepage]: https://github.com/osterman
[osterman_avatar]: https://github.com/osterman.png?size=150
[aknysh_homepage]: https://github.com/aknysh
[aknysh_avatar]: https://github.com/aknysh.png?size=150
[goruha_homepage]: https://github.com/goruha
[goruha_avatar]: https://github.com/goruha.png?size=150
[yufukui-m_homepage]: https://github.com/yufukui-m
[yufukui-m_avatar]: https://github.com/yufukui-m.png?size=150
[erik_img]: http://s.gravatar.com/avatar/88c480d4f73b813904e00a5695a454cb?s=144
[erik_web]: https://github.com/osterman/
[andriy_img]: https://avatars0.githubusercontent.com/u/7356997?v=4&u=ed9ce1c9151d552d985bdf5546772e14ef7ab617&s=144
[andriy_web]: https://github.com/aknysh/

184
README.yaml Normal file
View File

@ -0,0 +1,184 @@
# Name of this project
name: prometheus-to-cloudwatch
# License of this project
license: APACHE2
# Canonical GitHub repo
github_repo: cloudposse/prometheus-to-cloudwatch
# Logo for this project
#logo: docs/logo.png
# Badges to display
badges:
- name: "Build Status"
image: "https://travis-ci.org/cloudposse/prometheus-to-cloudwatch.svg?branch=master"
url: "https://travis-ci.org/cloudposse/prometheus-to-cloudwatch"
- name: "Latest Release"
image: "https://img.shields.io/github/release/cloudposse/prometheus-to-cloudwatch.svg"
url: "https://github.com/cloudposse/prometheus-to-cloudwatch/releases/latest"
- name: "Slack Community"
image: "https://slack.cloudposse.com/badge.svg"
url: "https://slack.cloudposse.com"
related:
- name: "Prometheus Operator"
description: "Prometheus Operator creates/configures/manages Prometheus clusters atop Kubernetes"
url: "https://github.com/cloudposse/prometheus-operator"
- name: "terraform-aws-cloudwatch-logs"
description: "Terraform Module to Provide a CloudWatch Logs Endpoint"
url: "https://github.com/cloudposse/terraform-aws-cloudwatch-logs"
- name: "terraform-aws-ecs-web-app"
description: "Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more."
url: "https://github.com/cloudposse/terraform-aws-ecs-web-app"
description: |-
Utility for scraping Prometheus metrics from a Prometheus client endpoint and publishing them to CloudWatch
usage: |-
__NOTE__: The module accepts parameters as command-line arguments or as ENV variables (or any combination of command-line arguments and ENV vars).
Command-line arguments take precedence over ENV vars
| Command-line argument | ENV var | Description |
|:-----------------------------|:-----------------------------|:------------------------------------------------------------------------------|
| aws_access_key_id | AWS_ACCESS_KEY_ID | AWS access key Id with permissions to publish CloudWatch metrics |
| aws_secret_access_key | AWS_SECRET_ACCESS_KEY | AWS secret access key with permissions to publish CloudWatch metrics |
| cloudwatch_namespace | CLOUDWATCH_NAMESPACE | CloudWatch Namespace |
| cloudwatch_region | CLOUDWATCH_REGION | CloudWatch AWS Region |
| cloudwatch_publish_timeout | CLOUDWATCH_PUBLISH_TIMEOUT | CloudWatch publish timeout in seconds |
| prometheus_scrape_interval | PROMETHEUS_SCRAPE_INTERVAL | Prometheus scrape interval in seconds |
| prometheus_scrape_url | PROMETHEUS_SCRAPE_URL | The URL to scrape Prometheus metrics from |
| cert_path | CERT_PATH | Path to SSL Certificate file (when using SSL for `prometheus_scrape_url`) |
| keyPath | KEY_PATH | Path to Key file (when using SSL for `prometheus_scrape_url`) |
| accept_invalid_cert | ACCEPT_INVALID_CERT | Accept any certificate during TLS handshake. Insecure, use only for testing |
| additional_dimension | ADDITIONAL_DIMENSION | Additional dimension specified by NAME=VALUE |
__NOTE__: If AWS credentials are not provided in the command-line arguments (`aws_access_key_id` and `aws_secret_access_key`)
or ENV variables (`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`),
the chain of credential providers will search for credentials in the shared credential file and EC2 Instance Roles.
This is useful when deploying the module in AWS on Kubernetes with [`kube2iam`](https://github.com/jtblin/kube2iam),
which will provide IAM credentials to containers running inside a Kubernetes cluster, allowing the module to assume an IAM Role with permissions
to publish metrics to CloudWatch.
examples: |-
### Build Go program
```sh
go get
CGO_ENABLED=0 go build -v -o "./dist/bin/prometheus-to-cloudwatch" *.go
```
### Run locally
```sh
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
export CLOUDWATCH_NAMESPACE=kube-state-metrics
export CLOUDWATCH_REGION=us-east-1
export CLOUDWATCH_PUBLISH_TIMEOUT=5
export PROMETHEUS_SCRAPE_INTERVAL=30
export PROMETHEUS_SCRAPE_URL=http://xxxxxxxxxxxx:8080/metrics
export CERT_PATH=""
export KEY_PATH=""
export ACCEPT_INVALID_CERT=true
./dist/bin/prometheus-to-cloudwatch
```
### Build Docker image
__NOTE__: it will download all `Go` dependencies and then build the program inside the container (see [`Dockerfile`](Dockerfile))
```sh
docker build --tag prometheus-to-cloudwatch --no-cache=true .
```
### Run in a Docker container
```sh
docker run -i --rm \
-e AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXXXXX \
-e AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX \
-e CLOUDWATCH_NAMESPACE=kube-state-metrics \
-e CLOUDWATCH_REGION=us-east-1 \
-e CLOUDWATCH_PUBLISH_TIMEOUT=5 \
-e PROMETHEUS_SCRAPE_INTERVAL=30 \
-e PROMETHEUS_SCRAPE_URL=http://xxxxxxxxxxxx:8080/metrics \
-e CERT_PATH="" \
-e KEY_PATH="" \
-e ACCEPT_INVALID_CERT=true \
prometheus-to-cloudwatch
```
### Run on Kubernetes
To run on `Kubernetes`, we will deploy two [`Helm`](https://helm.sh/) [charts](https://docs.helm.sh/developing_charts/)
1. [kube-state-metrics](https://github.com/kubernetes/charts/tree/master/stable/kube-state-metrics) - to generates metrics about the state of various objects inside the cluster, such as deployments, nodes and pods
2. [prometheus-to-cloudwatch](chart) - to scrape metrics from `kube-state-metrics` and publish them to CloudWatch
Install `kube-state-metrics` chart
```sh
helm install stable/kube-state-metrics
```
Find the running services
```sh
kubectl get services
```
Copy the name of the `kube-state-metrics` service (e.g. `gauche-turtle-kube-state-metrics`) into the ENV var `PROMETHEUS_SCRAPE_URL` in [values.yaml](chart/values.yaml).
It should look like this:
```sh
PROMETHEUS_SCRAPE_URL: "http://gauche-turtle-kube-state-metrics:8080/metrics"
```
Deploy `prometheus-to-cloudwatch` chart
```sh
cd chart
helm install .
```
`prometheus-to-cloudwatch` will start scraping the `/metrics` endpoint of the `kube-state-metrics` service and send the Prometheus metrics to CloudWatch
![kube-state-metrics-to-cloudwatch](images/kube-state-metrics-to-cloudwatch.png)
screenshots:
- name: "kube-state-metrics-service"
url: "images/kube-state-metrics-service.png"
description: "Kube state metrics"
- name: "kube-state-metrics-to-cloudwatch"
url: "images/kube-state-metrics-to-cloudwatch.png"
description: "Kube state metrics to cloudwatch"
# Contributors to this project
contributors:
- name: "Erik Osterman"
github: "osterman"
- name: "Andriy Knysh"
github: "aknysh"
- name: "Igor Rodionov"
github: "goruha"
- name: "yufukui-m"
github: "yufukui-m"

4
chart/Chart.yaml
View File

@ -1,5 +1,5 @@
apiVersion: v1
appVersion: "1.0"
appVersion: 0.1.2
description: A Helm chart for prometheus-to-cloudwatch
name: prometheus-to-cloudwatch
version: 0.1.0
version: 0.1.2

BIN
chart/charts/common-0.0.1.tgz
View File

Binary file not shown.

6
chart/requirements.lock
View File

@ -1,6 +0,0 @@
dependencies:
- name: common
repository: https://kubernetes-charts-incubator.storage.googleapis.com/
version: 0.0.1
digest: sha256:306378be27e855f6823d057c0d90ba3be227d44dc75b2c70c484a03c2515511e
generated: 2018-03-21T15:44:54.30424-04:00

2
chart/requirements.yaml
View File

@ -1,4 +1,4 @@
dependencies:
- name: common
version: 0.0.1
version: 0.0.4
repository: https://kubernetes-charts-incubator.storage.googleapis.com/

6
chart/values.yaml
View File

@ -1,4 +1,4 @@
# Default values for prometheus-to-cloudwatch
# Default values for prometheus-to-cloudwatch. Update them per your requirements
#
replicaCount: 1
@ -18,8 +18,8 @@ secrets:
image:
repository: "cloudposse/prometheus-to-cloudwatch"
tag: "0.1.0"
pullPolicy: "IfNotPresent"
tag: "0.1.2"
pullPolicy: "Always"
annotations: {}