[ad_1]
VMware Cloud Director allows Kubernetes-as-a-Service utilizing Container Service Extension(CSE). Suppliers can supply production-ready Tanzu Kubernetes Grid-based Kubernetes clusters utilizing Container Service Extension. The Container Service extension helps VMware Cloud Director with Multi-tenancy on the core, preserving Cloud Suppliers person personas in consideration. Builders in buyer organizations can set up and configure Tanzu packages into Tanzu Kubernetes clusters. This weblog describes how buyer group’s Kubernetes directors, or customers (Consumer Personas) can set up Consumer-managed Packages utilizing Tanzu CLI into CSE Managed Tanzu Kubernetes clusters. Desk 1 describes supported Consumer-Managed packages and their use circumstances on Tanzu Kubernetes Cluster. Desk 2 describes pre-installed core packages on Tanzu Kubernetes Cluster provisioned by CSE throughout the buyer group.
| Consumer Managed Bundle Title | Perform | Dependency | Set up Location |
|---|---|---|---|
| Cert-management | Certificates administration | required by Contour, Harbor, Prometheus, Grafana | Workload cluster or Shared Providers Cluster |
| Contour | Ingress management | required by Harbor, Grafana | Workload or Shared Providers Cluster |
| Harbor Registry | Container registry | n/a | Shared Providers Cluster |
| Prometheus | Monitoring | required by Prometheus | Workload Cluster |
| Grafana | Monitoring | n/a | Workload Cluster |
| Fluent-bit | Log forwarding | n/a | Workload Cluster |
| Bundle | Bundle Namespace | Description |
|---|---|---|
| antrea | tkg-system | Allows pod networking and enforces community insurance policies for Kubernetes clusters. If Antrea is chosen because the CNI supplier, this bundle is put in in each cluster |
| core-dns | tkg-system | Supplies DNS Service, put in in each cluster |
| vcd-csi | tkg-system | Supplies the VMware Cloud Director- Cloud Storage Interface. This bundle is put in in each cluster |
| vcd-cpi | tkg-system | Supplies the VMware Cloud Director- Cloud Supplier Interface. This bundle is put in in each cluster |
The set up of the Tanzu Bundle is a 3 steps course of that will likely be described on this article. Nevertheless, earlier than we deep dive into this, it’s assumed that the person function following these steps have desired rights to create the Kubernetes cluster on VMware Cloud Director and has put in kubectl on the native machine to handle the tanzu Kubernetes cluster. Moreover, you will need to notice that this weblog publish describes Tanzu Packages for Tanzu Kubernetes Grid model 1.4.0, VMware Cloud Director 10.3.2, and CSE launch 3.1.2.
This ought to be relevant for different Tanzu Kubernetes Grid variations so long as the TKG OVA model, utilized in step 1, matches with the bundle repository configured in step 2.
- Create and Put together Tanzu Kubernetes Cluster
- Put together Native Setting with Tanzu CLI and Tanzu Packages
- Set up Consumer Packages
1. Create and Put together Tanzu Kubernetes Cluster
Create Tanzu Kubernetes cluster utilizing VMware Cloud Director’s Kubernetes Container UI plug-in. The diagram showcases the abstract of Tanzu Kubernetes Cluster.
Alternatively, create Tanzu Kubernetes cluster utilizing YAML specification with vcd cli, as proven in snippet.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
|
$vi tkgm.yaml
apiVersion: cse.vmware.com/v2.0 sort: TKGm metadata: title: <span model=“background-color: rgba(0, 0, 0, 0.2); font-size: inherit; shade: preliminary;”>myk8scluster6</span> orgName: organization_name website: https://vcd_site_fqdn virtualDataCenterName: org_virtual_data_center_name spec: distribution: templateName: ubuntu–2004–kube–v1.21.2+vmware.1–tkg.1–7832907791984498322 settings: community: expose: true pods: cidr_blocks: – 100.96.0.0/11 companies: cidr_blocks: – 100.64.0.0/13 ovdcNetwork: ovdc_network_name rollbackOnFailure: true sshKey: null topology: controlPlane: depend: 1 cpu: null reminiscence: null sizingClass: Large_sizing_policy_name storageProfile: Gold_storage_profile_name employees: depend: 2 cpu: null reminiscence: null sizingClass: Medium_sizing_policy_name storageProfile: Silver_storage_profile
$ vcd cse cluster apply tkgm.yaml |
After the cluster creation is profitable, it ought to be just like the Determine 2 display screen.
Obtain kubeconfig from the VMware Cloud Director Buyer portal. As proven in determine 2, click on on ‘obtain kubeconfig’ to get the configuration file.
This file can be utilized because the kubectl present context by configuring the KUBECONFIG atmosphere variable as observe:
|
|
# PowerShell $ENV:KUBECONFIG=“C:UsersAdministratorDownloadskubeconfig-mycluster6.txt”
# Bash export KUBECONFIG=/root/kubeconfig–mycluster6.txt |
Alternatively, the kubeconfig file will be retrieved utilizing the cse-cli:
|
|
$ vcd cse cluster config mycluster6 > ~/.kube/config |
All the next instructions documented on this article assumes that the kubeconfig pointing to your cluster has been correctly setup. Run the next command to listing the cluster nodes and ensure kubeconfig is working:
Create Default Storage Class
Many of the packages that use a persistent quantity declare count on a default storage class. This file reveals an instance CSI storage class. You’ll must create your individual that displays your atmosphere, be certain that the annotation storageclass.kubernetes.io/is-default-class is about to true.
|
|
kubectl apply –f storage–class.yaml |
Confirm the CPI plugin used on the TKG cluster, and confirm the model is 1.1.0. The Steps to replace the CPI plugin are described right here.
Set up kapp-controller:
Tanzu Kubernetes Grid makes use of kapp-controller, it’s a bundle supervisor that handles the lifecycle of packages. It must be put in first by making use of the kapp-controller.yaml file connected beneath.
Extra particulars are within the VMware documentation.
|
|
kubectl apply –f kapp–controller.yaml |
As soon as the kapp-controller is put in, packages will be put in within the Tanzu Kubernetes Grid cluster by means of the Tanzu CLI.
2. Put together Native Setting with Tanzu CLI and packages
Set up Tanzu CLI 1.4 on the identical native machine by following the VMware documentation. The Tanzu CLI bundle plugin makes use of the kubectl present context so be certain that the precise context is configured earlier than operating tanzu CLI instructions.
Set up Tanzu CLI plugin for packages
As soon as the Tanzu CLI has been put in, if not already completed, be certain that the bundle plugin is put in or set up it as observe:
|
|
tanzu plugin set up —native cli bundle |
Add VMware Repositories
After the Tanzu CLI and its bundle plugin are put in, it’s essential so as to add repositories, one for the user-managed packages and one for the core packages.
The URL for these repositories displays the model of TKG that’s getting used for user-managed packages. For core packages it pertains to the TKG model and Kubernetes model. These variations should match the OVA model that has been used to deploy the cluster at step 1.
| TKG Model | Tanzu Normal Repository | Core Repository (per Kubernetes model) |
| 1.4.0 | v1.4.0 | v1.19.12_vmware.1-tkg.1 v1.20.8_vmware.1-tkg.2 v1.21.2_vmware.1-tkg.1 |
| 1.4.1 | v1.4.1 | v1.21.2_vmware.1-tkg.2 v1.20.8_vmware.1-tkg.3 v1.19.12_vmware.1-tkg.2 |
| 1.4.2 | v1.4.2 | v1.21.8_vmware.1-tkg.2 v1.19.16_vmware.1-tkg.1 v1.20.14_vmware.1-tkg.2 |
| 1.5.2 | v1.5.2 | v1.22.5_vmware.1-tkg.4 v1.20.14_vmware.1-tkg.5 v1.21.8_vmware.1-tkg.5 |
The repositories are added by operating the next instructions, notice the tag (after the colon) that should replicate the TKG model and Kubernetes model the cluster has been deployed with.
In our instance, we used the OVA for Kubernetes 1.21 from TKG 1.4.0 that interprets within the following instructions:
|
|
tanzu bundle repository add tanzu–customary —namespace tanzu–bundle–repo–world —create–namespace —url tasks.registry.vmware.com/tkg/packages/customary/repo:v1.4.0
tanzu bundle repository add tanzu–core —namespace tkg–system —create–namespace —url tasks.registry.vmware.com/tkg/packages/core/repo:v1.21.2_vmware.1–tkg.1 |
Record and set up the packages
As soon as the synchronization activity is accomplished, all of the out there packages will be listed as observe:
|
|
tanzu bundle out there listing –A |
For a selected bundle, the listing of obtainable model will be discovered as observe:
instance with Harbor bundle
|
|
tanzu bundle out there listing harbor.tanzu.vmware.com –A |
3. Set up Consumer-Managed packages
All of the examples within the following steps use the variations of the bundle for TKG 1.4.0. For different TKG model, you may want to make use of different packages variations. The namespaces used to put in the packages replicate the usual namespaces naming conference from Tanzu Kubernetes Grid.
Bundle Knowledge Values
Some Consumer-Managed packages require a data-values.yaml to be offered that may include configuration data associated to your atmosphere.
Please evaluate VMware documentation to grasp methods to retrieve the data-values template.
Set up Cert-Supervisor
Cert-manager offers a technique to simply generates new certificates and is required by many of the different packages. Use the next command to put in cert-manager.
Extra particulars within the VMware documentation.
|
|
tanzu bundle set up cert–supervisor —namespace cert–supervisor —create–namespace —bundle–title cert–supervisor.tanzu.vmware.com —model 1.1.0+vmware.1–tkg.2 |
Set up Contour
The Contour bundle is utilized by Harbor and Grafana.
As soon as the contour-data-values.yaml file has been created (this pattern can be utilized as is) use the next command to put in Contour.
Extra particulars within the VMware documentation.
|
|
tanzu bundle set up contour —namespace tanzu–system–contour —create–namespace —bundle–title contour.tanzu.vmware.com —model 1.17.1+vmware.1–tkg.1 —values–file contour–information–values.yaml |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
# Pattern of the file contour-data-values.yaml namespace: tanzu–system–ingress contour: configFileContents: {} useProxyProtocol: false replicas: 2 pspNames: “vmware-system-restricted” logLevel: information envoy: service: kind: LoadBalancer annotations: {} nodePorts: http: null https: null externalTrafficPolicy: Cluster disableWait: false hostPorts: allow: true http: 80 https: 443 hostNetwork: false terminationGracePeriodSeconds: 300 logLevel: information pspNames: null certificates: length: 8760h renewBefore: 360h |
Set up Harbor
As soon as the harbor-data-values.yaml file has been created (this pattern must be tweaked with at the very least a unique hostname and completely different password and secret) use the next command to put in Contour.
Extra particulars are within the VMware documentation.
|
|
tanzu bundle set up harbor —namespace tanzu–system–registry —create–namespace —bundle–title harbor.tanzu.vmware.com —model 2.2.3+vmware.1–tkg.1 —values–file harbor–information–values.yaml |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85
|
# Pattern of the file harbor-data-values.yaml namespace: tanzu–system–registry hostname: harbor.tanzu.lab port: https: 443 logLevel: information tlsCertificate: tls.crt: tls.key: ca.crt: enableContourHttpProxy: true harborAdminPassword: VMware1! secretKey: vNrsg4KQeQYOQi1J database: password: 6D4E9CENc7KUvhSX core: replicas: 1 secret: lfi4oeZ8lNqaTD5O xsrfKey: TPXlSXBGqDs98VJY4veNiawO6bA0qlgQ jobservice: replicas: 1 secret: DwF6fxLfZHnWeQfl registry: replicas: 1 secret: 5uefzEyT0rr3yuCn notary: enabled: true trivy: enabled: true replicas: 1 gitHubToken: “” skipUpdate: false persistence: persistentVolumeClaim: registry: existingClaim: “” storageClass: “” subPath: “” accessMode: ReadWriteOnce dimension: 10Gi jobservice: existingClaim: “” storageClass: “” subPath: “” accessMode: ReadWriteOnce dimension: 1Gi database: existingClaim: “” storageClass: “” subPath: “” accessMode: ReadWriteOnce dimension: 1Gi redis: existingClaim: “” storageClass: “” subPath: “” accessMode: ReadWriteOnce dimension: 1Gi trivy: existingClaim: “” storageClass: “” subPath: “” accessMode: ReadWriteOnce dimension: 5Gi imageChartStorage: disableredirect: false kind: filesystem filesystem: rootdirectory: /storage proxy: httpProxy: httpsProxy: noProxy: 127.0.0.1,localhost,.native,.inner pspNames: null metrics: enabled: false core: path: /metrics port: 8001 registry: path: /metrics port: 8001 exporter: path: /metrics port: 8001 |
In TKG 1.4, there’s a identified situation with Harbor that requires to use this KB to repair the harbor-notary-signer pod that fails to start out.
|
|
# Content material of overlay-notary-signer-image-fix.yaml file #@ load(“@ytt:overlay”, “overlay”)
#@overlay/match by=overlay.and_op(overlay.subset({“sort”: “Deployment”}), overlay.subset({“metadata”: {“title”: “harbor-notary-signer”}})) — spec: template: spec: containers: #@overlay/match by=”title”,expects=”0+” – title: notary–signer picture: tasks.registry.vmware.com/tkg/harbor/notary–signer–photon@sha256:4dfbf3777c26c615acfb466b98033c0406766692e9c32f3bb08873a0295e24d1 |
|
|
kubectl —namespace tanzu–system–registry create secret generic harbor–notary–singer–picture–overlay –o yaml —dry–run=consumer —from–file=overlay–notary–signer–picture–repair.yaml | kubectl apply –f –
kubectl —namespace tanzu–system–registry annotate packageinstalls harbor ext.packaging.carvel.dev/ytt–paths–from–secret–title.0=harbor–notary–singer–picture–overlay |
Create DNS report
When Harbor is deployed utilizing Contour as ingress, it’s required to create a DNS report that maps the exterior IP deal with of the Envoy load balancer service to the hostname of Harbor (specified within the harbor-data-values.yaml file).
This command output the exterior ip deal with:
|
|
$ kubectl –n tanzu–system–ingress get svc NAME TYPE CLUSTER–IP EXTERNAL–IP PORT(S) AGE contour ClusterIP 100.66.207.167 <none> 8001/TCP 11m envoy LoadBalancer 100.67.12.12 172.16.x.x 80:31850/TCP,443:32135/TCP 11m |
Use the exterior IP deal with assigned to the envoy service and create a DNS report accordingly.
In our instance, this PowerShell command creates it.
|
|
Add–DnsServerResourceRecordA –Title “harbor” –ZoneName “tanzu.lab” –IPv4Address “172.16.x.x” |
Observability
Metric Server
The metric server is required for the profitable operation of Prometheus, Grafana, and Fluent-bit packages. Use the next command to put in the metrics server on the Tanzu Kubernetes cluster.
|
|
tanzu bundle set up metrics–server —namespace tkg–system —create–namespace —bundle–title metrics–server.tanzu.vmware.com —model 0.4.0+vmware.1–tkg.1 |
Prometheus
Tanzu Kubernetes Grid contains signed binaries for Prometheus which you can deploy on Tanzu Kubernetes clusters to observe cluster well being and companies.
Whereas the set up will be personalized, the default configuration ought to be sufficient for many of the circumstances.
Extra particulars within the VMware documentation.
|
|
tanzu bundle set up prometheus —namespace tanzu–system–monitoring —create–namespace —bundle–title prometheus.tanzu.vmware.com —model 2.27.0+vmware.1–tkg.1 |
Grafana
Grafana lets you visualize and analyze metrics information collected by Prometheus in your clusters. It comes with out-of-the-box TKG Kubernetes cluster monitoring dashboards.
As soon as the grafana-data-values.yaml has been created (this pattern must be tweaked with at the very least a unique virtual_host_fqdn and completely different admin_password) use the next command to put in Grafana.
Extra particulars within the VMware documentation
|
|
tanzu bundle set up grafana —namespace tanzu–system–dashboards —create–namespace —bundle–title grafana.tanzu.vmware.com —model 7.5.7+vmware.1–tkg.1 —values–file grafana–information–values.yaml |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72
|
# Pattern of grafana-data-values.yaml file namespace: tanzu–system–dashboards grafana: deployment: replicas: 1 containers: sources: {} podAnnotations: {} podLabels: {} k8sSidecar: containers: sources: {} service: kind: LoadBalancer port: 80 targetPort: 3000 labels: {} annotations: {} config: grafana_ini: | [analytics] check_for_updates = false [grafana_net] url = https://grafana.com [log] mode = console [paths] information = /var/lib/grafana/information logs = /var/log/grafana plugins = /var/lib/grafana/plugins provisioning = /and so forth/grafana/provisioning datasource_yaml: |– apiVersion: 1 datasources: – title: Prometheus kind: prometheus url: prometheus–server.tanzu–system–monitoring.svc.cluster.native entry: proxy isDefault: true dashboardProvider_yaml: |– apiVersion: 1 suppliers: – title: ‘sidecarDashboardProvider’ orgId: 1 folder: ” folderUid: ” kind: file disableDeletion: false updateIntervalSeconds: 10 allowUiUpdates: false choices: path: /tmp/dashboards foldersFromFilesStructure: true pvc: annotations: {} storageClassName: null accessMode: ReadWriteOnce storage: “2Gi” secret: kind: “Opaque” admin_user: “YWRtaW4=” admin_password: “Vk13YXJlMSE=” ingress: enabled: true virtual_host_fqdn: “grafana.tanzu.lab” prefix: “/” servicePort: 80 tlsCertificate: tls.crt: tls.key: ca.crt:
|
Create DNS report
Identical to the Harbor case, a DNS report is required and will level to the identical exterior IP deal with.
In our instance, this PowerShell command creates it:
|
|
Add–DnsServerResourceRecordA –Title “grafana” –ZoneName “tanzu.lab” –IPv4Address “172.16.x.x” |
Managing Cluster and Bundle lifecycle
Please discuss with the product interop matrix to pick supported bundle variations. Please discuss with the Tanzu CLI documentation for numerous operations corresponding to set up, replace, delete of a Tanzu bundle.
Observe: This weblog publish is co-authored with Sachi Bhatt
Additional Studying
CSE 3.1.2 Technical Overview Weblog
Container Service Extension Characteristic Friday Episode 68
[ad_2]
