Go to file

Haitao Pan 7a1b15416f add scripts/auto-fix-gitleaks.sh		2025-04-05 13:57:43 +08:00
.github/workflows	workflows/app-pipeline-keycloak-server: update	2025-01-21 11:47:01 +08:00
app	build(deps): bump path-to-regexp and express in /app/js-demo/backend	2025-03-28 12:30:09 +00:00
config	Merge branch 'main' into vpn-overlay	2025-04-05 10:57:46 +08:00
docs	README.md: update	2024-11-09 21:09:53 +08:00
iac_modules	add scripts/auto-fix-gitleaks.sh	2025-04-05 13:57:43 +08:00
scripts	add scripts/auto-fix-gitleaks.sh	2025-04-05 13:57:43 +08:00
Solutions	add scripts/auto-fix-gitleaks.sh	2025-04-05 13:57:43 +08:00
vpn-overlay	feat(vxlan): enhance setup_sit_vxlan.sh, add overlay_diag.sh and README	2025-04-05 11:46:19 +08:00
.gitignore	feat(ansible): add dynamic inventory and ansible.cfg	2025-04-05 12:38:31 +08:00
.gitmessage.txt	feat(iac): Refactor structure and support multi-environment config loading	2025-03-29 11:09:24 +08:00
.gitmodules	merged: observability, aigc_app_engine, containerd_platform, micro_service, sso_identity_provider, devops	2025-03-24 20:53:24 +08:00
ansible.cfg	feat(ansible): add dynamic inventory and ansible.cfg	2025-04-05 12:38:31 +08:00
LICENSE	LICENSE: update -> GPL V3	2023-05-17 21:27:43 +08:00
README.md	Update README.md	2025-04-03 15:24:32 +08:00
requirements.txt	feat(ansible): add dynamic inventory and ansible.cfg	2025-04-05 12:38:31 +08:00

README.md

Modern Container Application Reference Architecture

Welcome to the repository for the Modern Container Application Reference Architecture. This repository contains a comprehensive guide and reference architecture for building scalable, portable, resilient, and agile containerized applications. 一个基于 Pulumi + Ansible 的基础设施自动化项目模板，支持多环境部署（dev / staging / prod），实现从基础设施创建到主机配置的全流程自动化管理。

Overview

The project aims to create a multi-cloud environment that leverages containers for deploying modern applications. The key objective is to set up a unified authentication system using OIDC via Keycloak for AWS, GCP, Azure, GitHub, **Harbor ** and **Grafana **.

🚀 项目功能

使用 Pulumi（Python）创建 AWS 基础设施（VPC、子网、安全组、EC2）
配置结构模块化：base.yaml, vpc.yaml, firewall.yaml, instances.yaml
支持 Spot / On-Demand 实例，支持 TTL 标签
自动输出 EC2 IP，动态生成 Ansible Inventory
使用 Ansible Playbook 远程安装软件或部署服务
支持多环境 stack（dev/staging/prod）

项目结构

├── config/                  # 多环境配置
│   ├── base.yaml
│   ├── vpc.yaml
│   ├── firewall.yaml
│   └── instances.yaml
├── iac_modules/
│   └── pulumi/
│       ├── deploy.py              # Pulumi 主入口
│       ├── modules/               # VPC/SG/EC2 模块
│       ├── utils/config_loader.py
│       └── requirements.txt
├── scripts/
│   ├── infra.sh                   # 一键部署脚本
│   └── inventory.py              # 动态 Ansible inventory
├── ansible/
│   └── playbooks/
│       └── setup.yml             # 应用部署 playbook

In this first phase, we focus on implementing OpenID Connect (OIDC) login functionality for the following platforms:

Cloud Resouces

Cloud	Host Name	Public IP	VPN IP	Costs	Service
ALI	cn-gateway.svc.plus	8.130.10.142	10.254.0.1	---	cn-proxy.onwalk.net
HW	hw-node.svc.plus	139.9.139.22	---	---	images.onwalk.net
AWS	global-gateway.svc.plus	52.196.108.28	10.255.0.1	---	tky-connector.onwalk.net global-images.onwalk.net
AWS	us-gateway.svc.plus	54.183.199.99	---	---	us-connector.onwalk.net grafana.onwalk.net metrics.onwalk.net
AWS	canada-gateway.svc.plus	35.182.201.252	---	---	ca-connector.onwalk.net logs.onwalk.net

Key Components Overview

Component	Description	Tools/Technologies
1. LDP (Login Delegation Protocol)	Centralized authentication and Single Sign-On (SSO) using Auth0 by Okta for various platforms.	Auth0 by Okta, OIDC
2. IaC (Infrastructure as Code)	Infrastructure management and provisioning using automated tools.	Terraform, Pulumi
3. Monitoring	Comprehensive observability and monitoring for the application, including system metrics, network, and performance.	Grafana Cloud, Prometheus, DeepFlow, ClickHouse
4. Git Repository	Version control and source code management for the project.	GitHub
5. CI/CD (Continuous Integration/Delivery)	Automated build, test, and deployment pipelines.	GitHub Actions

key Components Service

Name	Domain	Version	Deploy	Docker Compose	Chart	CI/CD
1. Keycloak	keycloak.onwalk.net	26.0	Docker	Yes	Yes	GitHub Actions
2. Harbor	images.onwalk.net	2.12	Docker	Yes	Yes	GitHub Actions
3. ChartMuseum	charts.onwalk.net	0.14.0	Docker	Yes	Yes	GitHub Actions
4. Vault	vault.onwalk.net	1.15	Docker	Yes	Yes	GitHub Actions
5. Nginx/OSS	mirrors.onwalk.net	1.21	Kubernetes	Yes	Yes	GitHub Actions

Name	Domain	**Version	Deploy	Docker Compose**	Chart	CI/CD
5. OpenIPA	freeipa.onwalk.net	4.10	Kubernetes, Docker, BareMetal	Yes	Yes	GitHub Actions
1. PostgreSQL	db.onwalk.net	16.0	Kubernetes, Docker	Yes	Yes	GitHub Actions
8. Prometheus	monitoring.onwalk.net	2.35	Kubernetes, Docker	Yes	Yes	GitHub Actions
9. Grafana	monitoring.onwalk.net	8.4	Kubernetes, Docker	Yes	Yes	GitHub Actions
10. Consul	consul.onwalk.net	1.12	Kubernetes, Docker	Yes	Yes	GitHub Actions
12. Jenkins	jenkins.onwalk.net	2.319	Kubernetes, Docker	Yes	Yes	GitHub Actions
13. GitLab	gitlab.onwalk.net	15.5	Kubernetes, Docker	Yes	Yes	GitHub Actions
14. MinIO	minio.onwalk.net	2023.2.0	Kubernetes, Docker	Yes	Yes	GitHub Actions

Set up Auth0 by Okta as the identity provider to enable OpenID Connect (OIDC) login for multiple platforms:
- AWS, GCP, Azure, GitHub, Grafana Cloud
OIDC allows secure Single Sign-On (SSO) across all these platforms.
For more details, refer to Platform-Specific OIDC Setup Docs.

2. IaC (Infrastructure as Code) - Terraform / Pulumi

Infrastructure for AWS, GCP, and Azure is provisioned using Terraform and Pulumi scripts.
These scripts allow easy and reproducible deployment and management of cloud resources.
See the iac/ folder for the setup files.

3. Monitoring - Grafana Cloud / Prometheus / DeepFlow / ClickHouse

Monitoring stack includes:
- Prometheus for metrics collection.
- DeepFlow for network and system observability.
- ClickHouse for storing and querying large amounts of observability data.
- Grafana Cloud for visualizing all collected metrics and logs.
Configuration files for monitoring tools can be found in the monitoring/ folder.

4. Git Repository - GitHub

All project code, infrastructure configurations, and documentation are managed within this GitHub repository.
GitHub also integrates with GitHub Actions for CI/CD.

5. CI/CD - GitHub Actions

Automated CI/CD pipeline is set up using GitHub Actions to ensure continuous integration and deployment.
Pipelines handle code testing, building, and multi-cloud deployments for platforms like AWS, GCP, and Azure.
YAML workflow files for GitHub Actions can be found in the .github/workflows/ directory.

For detailed instructions on configuring each platform, see:

TODO

Set up Auth0 by Okta as the identity provider for OIDC authentication.
Configure OIDC login for AWS.
Configure OIDC login for GCP.
Configure OIDC login for Azure.
Configure OIDC login for GitHub.
Configure OIDC login for Grafana Cloud.
Test and validate login workflows across all platforms.

Documentation

For more detailed information, please refer to the documentation available in two languages:

Getting Started

git submodule add --force https://github.com/svc-design/ansible.git git submodule add --force https://github.com/svc-design/iac_modules.git git submodule init git submodule update

New Ideas

CI Code Stages

ctags
https://github.com/KDAB/codebrowser
SonarQube
gitleaks

CD Deploy Stages

IAC
ArgoCD
Ansible Playbook

Follow the links above to the documentation in your preferred language to get started with using this reference architecture.

Contributing

We welcome contributions to this project. If you have suggestions, improvements, or find any issues, feel free to submit a pull request.

License

This project is released under the GPL V3 license. For more details, see the LICENSE file.

README.md Unescape Escape