My Tekton-CI project is coming along slowly but surely.

As a spare-time project, it is the thing I work on when I want to refresh my creative juices, or to explore an idea from things I’ve worked on before.

I’ve added a couple of new features this week:

Test Matrices

This is a simple concept, provide a list of options for a task, and execute the same task for each option.

My initial goal for this is test parallelism, by allowing multiple tasks to break up a step easily.

execute_tests:
  stage: test
  script:
    - ./run_tests.sh $TESTS_TO_RUN
  tekton:
    jobs:
      - TESTS_TO_RUN=integration
      - TESTS_TO_RUN=unit
      - TESTS_TO_RUN=e2e

This configuration would cause the ./run_tests.sh script to be run as three separate tasks, in parallel.

The first would have TESTS_TO_RUN=integration in the environment, the second would have TESTS_TO_RUN=unit, and finally the third would have TESTS_TO_RUN=e2e.

So, the ./run_tests.sh script could easily check to see what $TESTS_TO_RUN was, and do whatever is appropriate.

Successive tasks will wait for all the tasks created by the matrix to execute.

These values are merged to the variables environment variables.

NOTE: For now, this only works with scripts, but I’ll get it working with the Tekton Task integration, the matrix values will be passed in as an additional parameter to the task.

I have a feeling this is probably limited to a single node, because of the use of a PVC which will limit the level of parallelisation.

This might involve identifying the matrix elements up front, before creating the tasks, and spawning multiple parallel “task-pipelines” within the pipelinerun, each getting a volume-claim, and repeating the same tasks in each.

Knapsack

This would also be usable with Knapsack, with something like this (assuming you have rspec and Knapsack configured in your Gemfile).

variables:
  CI_NODE_TOTAL: "2"

execute_tests:
  stage: test
  script:
    - bundle exec rspec
  tekton:
    jobs:
      - CI_NODE=0
      - CI_NODE=1

This would allow Knapsack to split your tests across the different tasks automatically, thus shaving some time off your test suite.

GitHub Webhook Secrets

This is one major step towards a Beta-release of this code.

GitHub (and GitLab) can be configured to sign hook requests with a shared-secret, and it’s definitely recommended to validate this.

The current implementation uses a Secret in the same namespace as the container, called tekton-ci-hook-secrets.

$ kubectl create secret generic tekton-ci-hook-secrets --from-literal=bigkevmcd_tekton-ci=testsecret

When a hook comes in, the repo is looked up as a key in this secret, and the resulting string is used to confirm the signature in the request.

If the signature doesn’t match, the request is dropped.

Something I learned here, the keys in a Secret are very limited in what they’ll accept as characters.

The key that is looked for in the Secret, is the Repository’s “fullname”, e.g. “bigkevmcd/tekton-ci”, but with all / characters replaced with _, thus in this case, it would look for a key bigkevmcd_tekton-ci.

Improved observability

I’ve started adding crucial observability to the service, both metrics through Prometheus and logging, and I’ll continue to add new elements here.