My Tekton-CI project is coming along slowly but surely.
As a spare-time project, it is the thing I work on when I want to refresh my creative juices, or to explore an idea from things I’ve worked on before.
I’ve added a couple of new features this week:
This is a simple concept, provide a list of options for a task, and execute the same task for each option.
My initial goal for this is test parallelism, by allowing multiple tasks to break up a step easily.
execute_tests: stage: test script: - ./run_tests.sh $TESTS_TO_RUN tekton: jobs: - TESTS_TO_RUN=integration - TESTS_TO_RUN=unit - TESTS_TO_RUN=e2e
This configuration would cause the
./run_tests.sh script to be run as three
separate tasks, in parallel.
The first would have
TESTS_TO_RUN=integration in the environment, the second
TESTS_TO_RUN=unit, and finally the third would have
./run_tests.sh script could easily check to see what
was, and do whatever is appropriate.
Successive tasks will wait for all the tasks created by the matrix to execute.
These values are merged to the
variables environment variables.
NOTE: For now, this only works with scripts, but I’ll get it working with the Tekton Task integration, the matrix values will be passed in as an additional parameter to the task.
I have a feeling this is probably limited to a single node, because of the use of a PVC which will limit the level of parallelisation.
This might involve identifying the matrix elements up front, before creating the tasks, and spawning multiple parallel “task-pipelines” within the pipelinerun, each getting a volume-claim, and repeating the same tasks in each.
This would also be usable with Knapsack, with something like this (assuming you have rspec and Knapsack configured in your Gemfile).
variables: CI_NODE_TOTAL: "2" execute_tests: stage: test script: - bundle exec rspec tekton: jobs: - CI_NODE=0 - CI_NODE=1
This would allow Knapsack to split your tests across the different tasks automatically, thus shaving some time off your test suite.
GitHub Webhook Secrets
This is one major step towards a Beta-release of this code.
GitHub (and GitLab) can be configured to sign hook requests with a shared-secret, and it’s definitely recommended to validate this.
The current implementation uses a Secret in the same namespace as the container, called
$ kubectl create secret generic tekton-ci-hook-secrets --from-literal=bigkevmcd_tekton-ci=testsecret
When a hook comes in, the repo is looked up as a key in this secret, and the resulting string is used to confirm the signature in the request.
If the signature doesn’t match, the request is dropped.
Something I learned here, the keys in a Secret are very limited in what they’ll accept as characters.
The key that is looked for in the Secret, is the Repository’s “fullname”, e.g.
“bigkevmcd/tekton-ci”, but with all
/ characters replaced with
_, thus in
this case, it would look for a key
I’ve started adding crucial observability to the service, both metrics through Prometheus and logging, and I’ll continue to add new elements here.