This workflow is something that I was introduced to by a friend of mine – Mostapha. Obviously, I have been using GitHub to store and manage my code for quite some time, but I haven’t really been taking advantage of the new CI/CD tooling that GitHub now offers. One might ask why? Well, if you really think about it, I just didn’t need to.
My previous stints were at companies like Grimshaw or HOK, where I was the only one, or one of the very few, that worked on such tools. We had other people involved with these tools to some extent, but very rarely would we actually interact by working on the same code/tool. We have all been using GitHub there, but mainly to take advantage of its “Issues” tools. It was a good place for others to post feature requests or report bugs, so that I could work on them. What about releases, one might ask.
Well, we actually had a pretty good workflow in place. It was built around a tool, called Zombie. Zombie was a little app, that was downloading latest releases of our Revit add-ins from GitHub’s Releases, and deploying the latest version to the end user. All in the background. All quietly. All automatically. So, you might be asking yourself, how did the “Zombie workflow” work:
- I would work on the Revit add-in code and submit any changes to GitHub.
- When I had a new release ready, I would zip up all of my DLLs and create a new GitHub Release, upload the Zip assets to the release, and Zombie would take it from there.
- Zombie would ping GitHub Releases and check latest version installed versus one that was available. If we had a new version, it would download it, and unzip it to user’s add-ins directory.
The only non-automatic part of that workflow, was the fact that I had to create my Releases manually. I had to zip up my DLLs, and upload them to GitHub for this to work. That was a little tedious, but GitHub at the time (June/July 2018) didn’t offer any automation around that process. I think BitBucket had a nice CI/CD (continuous integration, continuous development) workflow built around a tool called Pipelines, but…I wasn’t on BitBucket. Anyways, it wasn’t until later that year (October 2018), that GitHub announced GitHub Actions and the problem would have been solved.
I haven’t really gotten around to using GitHub Actions until last year, when I was working on Honeybee for Revit. Since then, I really took to it, and quite like the workflow. I also like to combine it with Git Flow for managing branches, releases, fixes etc. Here’s what my typical workflow looks like nowadays:
- Host code on GitHub.
- Manager Git with Git Flow workflow.
- Implement CI/CD with GitHub Actions.
- Host shared resources on GitHub Packages…but we will talk about it at some other time.
Everyone knows GitHub, so I won’t go into it. The only thing I wanted to touch base on here, is why GitHub over something like GitLab or BitBucket. I have had these conversations with my managers before, and the issue usually came down to money. GitHub historically has been more expensive than BitBucket, and quite frankly, it had less CI/CD tools. It was always the more intuitive, more popular, more “open source” vibe platform than BitBucket. People wanting a simple, user friendly solution to host their OS project, would do so on GitHub. There was a huge OS community on GitHub, so it just felt right. If you were a solo practitioner, you would get unlimited public repos, a few private ones, and you were good to go. If you wanted to turn pro, needed private repos, and cheap option to add more developers, you were likely to switch to BitBucket. Nowadays, GitHub is cheaper than BitBucket, has as many if not more project management, CI/CD tools and most importantly for me, it “never” goes down. I can’t tell you how often we have experienced technical difficulties with BitBucket. It’s a no brainer to be on GitHub.*
*I get to use both since all of the project that I usually manage are on GitHub, while my colleague – Adam – hosts all of the ones that he manages on BitBucket. I get to experience both, and let me tell you – GitHub is nice!
OK, so you got your code uploaded to GitHub, what now? If you are the only one working on the code, you are probably not worried about branch management. However, just in case you have two(2) or three(3) other people working on it, and submitting changes, it’s time to implement some rules around branch management. Git is a great tool, but just like with any great tool, it takes some skill, experience, and most importantly a set of rules for how it will be used so that’s its used effectively and consistently. Git Flow is just that. It’s a simple wrapper around Git that helps you manage branches and keep everything organized. If you have installed Git on your machine, you also have the Git preferred command line tool called mingw-w64, as well as Git Flow plugin that comes with it. What I usually do, is call
git flow init on my current repository then set my settings to:
- release branch to `master`
- development branch to `develop`
- feature prefix to `feature/`
- bugfix prefix to `bugfix/`
- hotfix prefix to `hotfix/`
- release prefix to `release/`
- version tag prefix to `v` etc.
Once you have your Git Flow initiated, you can now manage the repo following this simple process described in detail here: Git Flow Workflow For me that usually means that I can start a new feature by calling
git flow feature start name-of-the-feature and git flow will take over from here. What it does is that it checks out the develop branch, creates a new branch from it called
feature/name-of-the-feature , sets it as the current branch, and you are good to start working. All of that happens automatically so I don’t have to call 3 or 4 different methods. I like that. Once I am done with the feature, I can call
git flow feature finish and Git Flow will merge my
feature/name-of-the-feature branch into
develop branch, it will then delete the remote
feature/name-of-the-feature branch as well as the local version of it, checkout
develop for me, and I am back on
develop ready to work on the next feature. What I like about this workflow, is that it removes the pieces of the Git management that are simply tedious. I don’t have to remove the remote/local branches when I am done with the feature. It saves me a few calls, and keeps my repo nice and tidy. Finally, when I am ready to publish my changes I would call
git flow release start 1.0.1 or any other version, and it handles all of the underlying branch management. Release branches get merged into
develop has master merged into it to keep them both in sync. It also creates release Tags and prefixes them with the
v so my version 1.0.1 tag looks like
v1.0.1. Why does it matter? We will talk about this next.
This is the piece that brings it all together for me. Once I am done with a given release, what I usually want to do at this point, is create a GitHub Release, and upload new assets to it so that my latest tool is available for download. Like I mentioned before, when I was at HOK, this step was executed manually, and took some time. Nowadays, I just setup a GitHub Action that looks like this:
What the above does for us, is that it will trigger GitHub to take our repository, checkout the default branch, restore all of the dependencies via Nuget, build projects like Revit 2020, Revit 2021, Setup (MSI) etc., create a new Release, and finally upload the MSI resource that was created in this process and upload it to the Release. Line 7 is what ties this Action workflow to our previous step which was to create a release branch via Git Flow. I am not sure if you remember, but we setup Git Flow to automatically prefix all new release tags with a
v prefix. That is then used to “trigger” GitHub Actions to fire this workflow off automatically. That’s pretty neat!
What about ZIPs? In the above workflow, part of building the Setup project is that it produces an MSI output. So we have access to the output of the project that is being built remotely. Why not ZIPs? We can do that as well. We can even go as far as calling any commands that are available from PowerShell, so we can create directories, copy files around, zip them etc. That’s nice! Here’s an example:
The above allows us to build the project, and then run a PowerShell routine to create a directory with a subdirectory, copy DLLs that were produced by building the project into it, then zipping it all up together, creating the release, and uploading the resulting ZIP file to the release as an asset. That would have been a perfect compliment to my old Zombie workflow. It would remove the manual step of having to create the GitHub Release and populating it with the ZIP-ed up assets.
One thing worth noting here is that the above workflow is completely feasible via the GitHub Actions, but would not be possible via BitBucket Pipelines. As far as I know – and please correct me if I am wrong – BitBucket Pipelines do not support .NET Framework projects, and only support .NET Core ones. In the Revit world, we are much more likely to be working with .NET Framework, than we would be with .NET Core. I think you might be able to achieve proper CI/CD on BitBucket side of things via Azure DevOps tool also called Pipelines instead of BitBucket’s Pipelines, but I am just a little skeptical about managing ever more complex technology stacks. The best technology stacks, in my opinion, are the simplest ones. If there was one thing in my architectural education that I have learned was that
Less is more
I hope the above is a good primer for people getting into coding, and looking for ways to automate their day to day branch management, release management and creation.
Ps. Zombie is an Open Source project, available for free from HOK. Please get it here!
Ps2. So many “pipelines”. Just to make sure no one confuses them, let’s make sure one thing is clear. There are two(2) tools called Pipelines:
- BitBucket Pipelines part of the Atlassian suite of tools.
- Azure Pipelines part of the Azure DevOps suite of tools.
Azure Pipelines built by Microsoft, and totally works well with .NET Core and .NET Framework projects.
BitBucket Pipelines built by Atlassian, and totally not my cup of tea.