Add lastly duties to Tekton pipelines – IBM Developer

Tekton Pipeline 0.14 launched the lastly clause within the Pipeline specification. This part is good for operating something simply earlier than exiting the pipeline, resembling cleansing up any acquired assets, sending notifications, rolling again deployments, and extra.

As outlined on the Tekton web site, the lastly part takes an inventory of a number of duties which are all executed in parallel in any case duties are completed executing — no matter successful or a failure. The next code exhibits the lastly part of a pipeline.

apiVersion: tekton.dev/v1beta1
sort: Pipeline
metadata:
  title:build-deploy-cleaup-pipeline
spec:
  params:
    - title: api-url
    - title: cloud-region
  duties:
    - title: clone
       taskRef:
         title: git-clone
    - title: construct
       taskRef:
         title: construct
       runAfter:
         - clone
    - title: deploy
       taskRef:
         title: deploy
       runAfter:
         - construct
  lastly:
    - title: cleanup
       taskRef:
         title: cleanup

Additionally, the execution standing of a lastly process impacts the general pipelineRun standing. A pipelineRun is asserted failed if a number of lastly duties fail.

The group has carried out many options over the previous few releases to assist design lastly duties for frequent use instances.

Use instances for lastly

A Tekton Pipeline is carried out as a Kubernetes controller and executes a set of pods in your Kubernetes cluster. You possibly can configure a pipeline utilizing a directed acyclic graph (DAG) and a set of lastly duties primarily based in your steady integration and steady supply (CI/CD) workflow. The most typical workflow for CI/CD is build-test-deploy, which we lengthen right here to incorporate lastly eventualities.

Ship notification

As the next picture exhibits, a CI/CD pipeline defines a lastly process named send-notification that references send-to-channel-slack and/or sendmail to inform the group and sends successful or failure notification relying on the execution standing of the construct course of. The construct course of contains construct and check duties within the duties part of the pipeline.

Cleanup assets

A few cleanup eventualities:

• A pipeline creates a venture or acquires assets and sends the title of the venture or assets to the lastly process. The lastly process can then clear up these venture assets no matter a failure within the duties part, as proven within the following picture:

  

• A pipeline provisions a brand new Kubernetes cluster. The pipeline begins executing and stops for some purpose. The pipeline has outlined lastly duties to unencumber the Kubernetes cluster. Although the pipelineRun stops, the lastly part executes to unencumber the assets.

Terminate reasonably than cancel

You might be operating a pipeline that executes a couple of Extract, Remodel, and Load (ETL) processes in a sequence, as proven within the following picture. Due to useful resource constraints, you need to cancel scheduling any additional processes however compelete executing the present operating course of and analyze the outcomes from that course of, as proven on this picture:

These are the most typical use instances that require the performance of lastly or an exit handler in a pipeline. Subsequent, let’s have a look at the record of the options that we launched to perform these use instances.

Use lastly with if

The lastly duties are assured to execute earlier than the pipelineRun exits. Nevertheless, there are use instances the place the pipeline wants the pliability to not run a lastly process primarily based on a sure situation (for instance, the person desires to ship a Slack notification provided that a construct fails). We’ve carried out two options to make this use case potential: TEP-0045 and TEP-0028 have the main points. The general concept right here is to allow specifying when expressions in a lastly process and supply a method to entry the execution standing of a process in a lastly process at runtime as a way to consider specified when expressions. You possibly can entry an execution standing of a process utilizing a context variable $(duties.<task-name>.standing):

spec:
  pipelineSpec:
    duties:
      - title: golang-build
        taskRef:
          title: golang-build
      - title: unit-test
        taskRef:
          title: golang-unit
        runAfter: [ golang-build ]
      - title: deploy
        taskRef:
          title: deploy
        runAfter: [ unit-test ]
    lastly:
      - title: notify-build-failure
        when:
          - enter: $(duties.golang-build.standing)
            operator: in
            values: ["Failed"]
        taskRef:
          title: send-to-slack-channel

Now, as an alternative of notifying just for a construct failure, you need to ship a Slack notification for any failure. TEP-0049 implements accessing an mixture standing of the duties part in lastly utilizing a context variable $(duties.standing):

spec:
  pipelineSpec:
    ...
    lastly:
      - title: notify-any-failure
        when:
          - enter: $(duties.standing)
            operator: in
            values: ["Failed"]
        taskRef:
          title: send-to-slack-channel

Join duties and at last sections

If you’re a pipeline writer, now you can ship the execution outcomes of a process to any lastly process in order that the venture assets created/acquired by the pipeline are assured to be cleaned up by the lastly process. TEP-0004 explains the design particulars of this characteristic.

spec:
 duties:
   - title: purchase
     taskRef:
       title: purchase
 lastly:
   - title: launch
     taskRef:
       title: launch
     params:
       - title: leased-resource
         worth:  $(duties.purchase.outcomes.leased-resource)

Gracefully terminate a pipeline

Rafal Bigraj spent a variety of effort in designing and implementing a characteristic (TEP-0058), which made one of many frequent machine studying use instances potential utilizing Tekton.

We carried out an answer to allow sleek termination (cancellation) of a pipeline the place the present operating duties are cancelled or terminated and the cleanup is scheduled. When a pipelineRun is executing, you possibly can replace its definition to cancel the duties, which deletes all of the respective pods and schedules the lastly part.

apiVersion: tekton.dev/v1beta1
sort: PipelineRun
metadata:
  title: etl-processes
spec:
  standing: "CancelledRunFinally"

We carried out another resolution the place the present operating duties will not be interrupted till they’re completed and no new duties are scheduled. After the present operating duties end, the lastly part is executed to research the outcomes from already executed duties.

apiVersion: tekton.dev/v1beta1
sort: PipelineRun
metadata:
  title: etl-processes
spec:
  standing: "StoppedRunFinally"

Now you know the way to design a lastly part of a pipeline to implement varied use instances. You probably have a use case or want a brand new characteristic in Tekton, attain out to the Tekton group over Slack or open a dialogue within the tektoncd/pipeline repo.