Managing a GKE Multi-tenant Cluster with Namespaces

admin

2 years ago

“Managing a GKE Multi-tenant Cluster with Namespaces”

Daftar Isi

Toggle

Pengantar

Saat mempertimbangkan solusi pengoptimalan biaya untuk setiap infrastruktur Google Cloud yang dibangun di sekitar cluster Google Kubernetes Engine (GKE), penting untuk memastikan bahwa Anda menggunakan resource yang ditagih secara efektif. Salah langkah umum adalah menetapkan rasio pengguna atau tim satu banding satu ke klaster, yang mengakibatkan proliferasi klaster.

Klaster multi-penyewa memungkinkan beberapa pengguna atau tim untuk berbagi satu klaster untuk beban kerja mereka sambil mempertahankan isolasi dan pembagian sumber daya yang adil. Ini dicapai dengan membuat ruang nama. Ruang nama memungkinkan beberapa cluster virtual ada di cluster fisik yang sama.

Praktikum

Task 1. Download required files

gsutil -m cp -r gs://spls/gsp766/gke-qwiklab ~
cd ~/gke-qwiklab

Task 2. View and create namespaces

setting zone

gcloud config set compute/zone us-central1-a && gcloud container clusters get-credentials multi-tenant-cluster

Default namespaces

Cek namespace

kubectl get namespace
kubectl api-resources --namespaced=true
kubectl get services --namespace=kube-system

Creating new namespaces

Buat namespaces baru

kubectl create namespace team-a && \
kubectl create namespace team-b

Deploy app

kubectl run app-server --image=centos --namespace=team-a -- sleep infinity && \
kubectl run app-server --image=centos --namespace=team-b -- sleep infinity

kubectl get pods -A

Cek detail pod

kubectl describe pod app-server --namespace=team-a

Setitng spesific namespace

kubectl config set-context --current --namespace=team-a
kubectl describe pod app-server

Task 3. Access Control in namespaces

Setting IAM -> tambahkan Role View untuk Kubernetes Engine Cluster

gcloud projects add-iam-policy-binding ${GOOGLE_CLOUD_PROJECT} \
--member=serviceAccount:team-a-dev@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com  \
--role=roles/container.clusterViewer

Kubernetes RBAC

setup single rule via kubectl

kubectl create role pod-reader \
--resource=pods --verb=watch --verb=get --verb=list

ssetup multi rule dengan yaml

nano developer-role.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: team-a
  name: developer
rules:
- apiGroups: [""]
  resources: ["pods", "services", "serviceaccounts"]
  verbs: ["update", "create", "delete", "get", "watch", "list"]
- apiGroups:["apps"]
  resources: ["deployments"]
  verbs: ["update", "create", "delete", "get", "watch", "list"]

Apply rule yaml

kubectl create -f developer-role.yaml

config binding

kubectl create rolebinding team-a-developers \
--role=developer --user=team-a-dev@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com

Test the rolebinding

gcloud iam service-accounts keys create /tmp/key.json --iam-account team-a-dev@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com

Aktive service account

gcloud auth activate-service-account  --key-file=/tmp/key.json

setup credensial untuk cluster

gcloud container clusters get-credentials multi-tenant-cluster --zone us-central1-a --project ${GOOGLE_CLOUD_PROJECT}

cek pod

kubectl get pods --namespace=team-a

renew credensial

gcloud container clusters get-credentials multi-tenant-cluster --zone us-central1-a --project ${GOOGLE_CLOUD_PROJECT}

Task 4. Resource quotas

Setting limit team-a -> pod : 2 , loadbalance : 1

kubectl create quota test-quota \
--hard=count/pods=2,count/services.loadbalancers=1 --namespace=team-a

Buat pod ke 2 untuk team-a

kubectl run app-server-2 --image=centos --namespace=team-a -- sleep infinity

Test buat pod ke 3

kubectl run app-server-3 --image=centos --namespace=team-a -- sleep infinity

Error

Error from server (Forbidden): pods "app-server-3" is forbidden: exceeded quota: test-quota, requested: count/pods=1, used: count/pods=2, limited: count/pods=2

Cek resource detail

kubectl describe quota test-quota --namespace=team-a

Update test-quota supaya bisa 6 pod

export KUBE_EDITOR="nano"
kubectl edit quota test-quota --namespace=team-a

edit bagian

spec:
  hard:
    count/pods: "6"

Update script

kubectl describe quota test-quota --namespace=team-a

CPU and memory quotas

nano cpu-mem-quota.yaml

apiVersion: v1
kind: ResourceQuota
metadata:
  name: cpu-mem-quota
  namespace: team-a
spec:
  hard:
    limits.cpu: "4"
    limits.memory: "12Gi"
    requests.cpu: "2"
    requests.memory: "8Gi"

apply config

kubectl create -f cpu-mem-quota.yaml

create demo app

nano cpu-mem-demo-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: cpu-mem-demo
  namespace: team-a
spec:
  containers:
  - name: cpu-mem-demo-ctr
    image: nginx
    resources:
      requests:
        cpu: "100m"
        memory: "128Mi"
      limits:
        cpu: "400m"
        memory: "512Mi"

Apply config

kubectl create -f cpu-mem-demo-pod.yaml --namespace=team-a
kubectl describe quota cpu-mem-quota --namespace=team-a

Task 5. Monitoring GKE and GKE usage metering

Baca Juga : VPC Network Peering

Monitoring Dashboard

Buka menu -> Operations > Monitoring
Pilih Dashboard
Pilih GKE
Klik View All
Pada ADD FILTER box pilih Namespaces > team-a
Klik Apply

Metrics Explorer

Pilih Metrics Explorer
Pada Resource & Metrics field , Pilih Kubernetes Container
Klik Kubernetes Container > Container
Pilih CPU usage time
Klik apply
untuk exclude kube-system namespace, clik Add a filter pada filter section
pilih namespace_name as a Label
Pilih != (does not equal)
Klik Done

GKE usage metering

gcloud container clusters \
update multi-tenant-cluster --zone us-central1-a \
--resource-usage-bigquery-dataset cluster_dataset

Create the GKE cost breakdown table

Setup path billing table

export GCP_BILLING_EXPORT_TABLE_FULL_PATH=${GOOGLE_CLOUD_PROJECT}.billing_dataset.gcp_billing_export_v1_xxxx
export USAGE_METERING_DATASET_ID=cluster_dataset
export COST_BREAKDOWN_TABLE_ID=usage_metering_cost_breakdown

download template

export USAGE_METERING_QUERY_TEMPLATE=~/gke-qwiklab/usage_metering_query_template.sql
export USAGE_METERING_QUERY=cost_breakdown_query.sql
export USAGE_METERING_START_DATE=2020-10-26

generate template

sed \
-e "s/\${fullGCPBillingExportTableID}/$GCP_BILLING_EXPORT_TABLE_FULL_PATH/" \
-e "s/\${projectID}/$GOOGLE_CLOUD_PROJECT/" \
-e "s/\${datasetID}/$USAGE_METERING_DATASET_ID/" \
-e "s/\${startDate}/$USAGE_METERING_START_DATE/" \
"$USAGE_METERING_QUERY_TEMPLATE" \
> "$USAGE_METERING_QUERY"

Setup cost breakdown

bq query \
--project_id=$GOOGLE_CLOUD_PROJECT \
--use_legacy_sql=false \
--destination_table=$USAGE_METERING_DATASET_ID.$COST_BREAKDOWN_TABLE_ID \
--schedule='every 24 hours' \
--display_name="GKE Usage Metering Cost Breakdown Scheduled Query" \
--replace=true \
"$(cat $USAGE_METERING_QUERY)"

Create the data source in Data Studio

Buka https://datastudio.google.com/c/navigation/datasources
klik Create > Data Source -> add new data storage
Klik Continue
Pilih BigQuery

Klik Authorize
rename namanya
Dari Colom pertama pilih CUSTOM QUERY
pilih project id

 SELECT *  FROM `[PROJECT-ID].cluster_dataset.usage_metering_cost_breakdown`

Klik connect

Klik CREATE REPORT

Klik ADD TO REPORT

Create a Data Studio Report

ganti value

Data Range Dimension: usage_start_time
Dimension: namespace
Metric: cost

Pada data panel click Add a Filter

Klik save
Click Add a Filter Lagi

Klik save

Klik kanan pada table yang sudah di buat -> klik Duplicate
Drag duplicate table object di manapun di area report
Klik Header

pilh pie chart

Hasilnya

Dari top toolbar click Add a chart dan pilih Donut untuk create donut chart
drag ke dalam report dan tambahkan konfigurasi berikut

Data Range Dimension: usage_start_time
Dimension: resource_name
Metric: cost

Klik Add a Filter , masukan 2 filter yg sebelumnya

Dari top toolbar click Add a control dan pilih Drop-down list.
drag ke donut chart , kemudian config

Data Range Dimension: usage_start_time
Control field: namespace
Metric: None

Klik add filter
pilih unallocated (namespace filter)
Klik kanan pilih group

Klik View

Dari share menu pilih Download report

Penutup

Sahabat Blog Learning & Doing demikianlah penjelasan mengenai Managing a GKE Multi-tenant Cluster with Namespaces. Semoga Bermanfaat . Sampai ketemu lagi di postingan berikut nya.