A couple of weeks ago, I missed a weekly post as my wife and I traveled out of town and celebrated having both moved 1 year past our respective surgeries. So, this week it’s time to pay down that debt with a bonus post! I’ll cover a power-user technique for GitHub Actions that can take your workflows to the next level. To explain the idea, I’ll start with a practical example of a common problem and how to solve it. I’ll then show you how the solution can be generalized to enable more dynamic workflows.
An environmental issue
It’s not uncommon to need environment variables in your CI/CD process. With GitHub Actions, we often put these at the workflow level or job level. This prevent us from needing to repeat those declarations in each step.
For more advanced situations, however, we may need to generate these values dynamically. For example, we may want to use the workflow state, a special file, or an external system to determine what values should be configured. To do this, we can take advantage of the
environment file. If you’re not familiar with this approach, each step in a GitHub workflow is provided with a special file that is used to add environment variables to all future steps. GitHub provides us with a path to the file in the special GITHUB_ENV environment variable. We just need to add a line for each new variable, like this:
1- name: Set environment variables
2 run: |
3 echo "MY_VAR=Hello, World!" >> $GITHUB_ENV
4 echo "A_SECOND_VAR=Also Hello, World!" >> $GITHUB_ENV Some tools – such as the Amazon AWS CLI – rely on environment variables for some of its settings. Being able to specify those dynamically or statically is therefore valuable. The problem happens if there is then a need to unset one of those variables. Once a variable is set for the job environment, there is no available GitHub command to unset it. As an example:
1- name: Unset environment variables
2 run: |
3 # This sets the value to an empty string,
4 # but the variable still exists
5 echo "MY_VAR=" >> $GITHUB_ENV
6
7 # This unsets the variable and removes it,
8 # but only for this single step.
9 unset A_SECOND_VAR Naturally, most people will try to override the variable with an empty value (or an expression that evaluates to nothing). In both cases, the result is the same: the variable still exists, but the value is an empty string.
1- name: Another unworkable solution
2 env:
3 A_SECOND_VAR: ""
4 run: |
5 # The value is set to an empty string,
6 # but the variable still exists
7 echo $A_SECOND_VAR The inability to remove an environment variable can cause unexpected behaviors with command line tools. If the application is expecting the environment variable, it usually assumes the provided value is not an empty string. If it is, the application can suddenly fail and exit with an error code.
So, how do we unset a value?
An objective solution
It turns out the solution is to use a similar technique to a dynamic matrix. We can build a JavaScript object. That object can then be used to help us configure the appropriate environment. How does this work?
The YAML we are using in our workflow is ultimately just a way of representing structured objects as text. While most examples show assigning a simple value to a given key, what we’re really doing is building an object. The YAML above creates a new object for the env key. It then adds a set of name-value properties to that object. It doesn’t require us to work that way, however.
If we create a new object in JavaScript, we can use the fromJson expression to convert a string value into a complete object. That object can then be assigned directly to env. For example:
1- id: setMyEnv
2 run: |
3 echo 'MY_OBJECT={ "A_SECOND_VAR": "Hello", "A_THIRD_VAR": "Hello again" }' >> $GITHUB_OUTPUT
4
5- name: Almost the solution
6 env: ${{ fromJson(steps.setMyEnv.outputs.MY_OBJECT) }}
7 run: |
8 # The value is set from the object, so we
9 # should see "Hello" printed here.
10 echo $A_SECOND_VAR In this example, we created a new output variable for the step called MY_OBJECT. So that we can read the value, the step is assigned an id, setMyEnv. In the next step, we then read the value using steps.setMyEnv.outputs.MY_OBJECT. This is a string value, but it’s a string representation of a JavaScript object. We can use the fromJson expression to convert that string into a JavaScript object. We then use the ${{ ... }} syntax to assign the result of that expression to the env key.
Of course, this still doesn’t unset any previous keys. It just configures the env key, which merges with job or workflow settings (and overwrites any existing keys). So, how do we unset a key?
To complete the solution, we have to avoid using the workflow-level or job-level environment settings for any keys that may need to be unset. It can still be used for values that should remain static, of course. From there, create the object dynamically so that you choose which, if any, environment variables will be created. Then, use the step output to set the environment variables collectively for any steps that will need these dynamic values.
In short, you set the mutable values using a step or job output. Making this work for an entire job requires you to first promote the step output to a job output:
1myjob:
2 # The type of runner that the job will run on
3 runs-on: ubuntu-latest
4 outputs:
5 ENVSETTINGS: ${{ steps.setMyEnv.outputs.MY_OBJECT }}Next, set that output as env value in any later job. To make that value, any job receiving these values will require a needs that points to the source job. For example:
1jobs:
2 myjob2:
3 needs: [ myjob ]
4 runs-on: ubuntu-latest
5 env: ${{ fromJson(needs.myjob.outputs.ENVSETTINGS) }}
6 steps:
7 - name: Show the value
8 run: |
9 echo $A_SECOND_VAR By storing the variables using a step or job output, we can dynamically build an object that has the exact configuration we need without needing to unset a value. This provides a robust way to take ownership over the environment variables we’re creating and providing, and it allows steps or jobs to be configured more discretely.
Expanding on the solution
So, it turns out this approach isn’t just limited to matrix jobs or env settings. It works for most of the settings in a workflow (aside from uses, which can never be dynamic). You can assign objects to many of the elements of your workflow. This can allow you to dynamically control the steps and Actions themselves! This doesn’t let you set objects that require static values – such as those with the uses key, but it can work in surprising places. What do I mean? Let me show you!
1- id: myvars
2 run: |
3 SETTINGS='{"set-safe-directory": "true", "fetch-depth": "2", "show-progress": "true" }'
4 echo "GIT_SETTINGS=$SETTINGS" >> $GITHUB_OUTPUT
5
6- uses: actions/checkout@v4
7 with: ${{ fromJSON(steps.myvars.outputs.GIT_SETTINGS) }}That’s right … I set all of the with block settings dynamically! When this code runs, you will notice that the settings are a mix of the defaults (when values are not set) and our dynamic settings:
1▼ Run actions/checkout@v4
2 with:
3 set-safe-directory: true
4 fetch-depth: 2
5 show-progress: trueThis is an incredibly powerful technique that can enable you to configure multiple settings at once. It can also give you a partial workaround for the fact that Actions doesn’t yet YAML anchors or aliases as a way to configure steps once and then reuse them. And all of this because the expression language allows us to assign values or objects to the keys (as appropriate). I hope that being aware of this opens up some new possibilities for you and your workflows.
Happy DevOp’ing!