No matter what language or package manager, dependency management in most projects suffer from many of the same problems:
Whether its npm, NuGet, PyPI, maven, Docker etc… it feels like a full-time job sometimes to be able to manage packages the way we all would like to. The problem is exacerbated depending on your company methodology for maintaining projects, the architecture of your repository granularity (i.e. do I have 100 different GIT repos to maintain packages for independently) and whether you live in a product-based or project-based world.
There must be a better way?
My first attempt towards solving the dependency management conundrums was two years ago. I started out specifically to automate some of the dependency upgrades within a .NET Core related GitHub project. This led me no further than a fantastic tool called “NuKeeper“. I’m a big NuKeeper fan. It has a ton of features that are second to none I’ve found in accomplishing some of the intricacies in dependency management that you’d want.
NuKeeper is a command line utility that will create Pull Requests based on proposed package updates with NuGet semantic versioning. As such this tool is specifically for .NET, but has fantastic support across Framework and Core (and several GIT providers).
I found its ability to configure includes and excludes based on wildcards incredibly helpful. Additionally, you can configure the granularity of Pull Requests that you want. For example, I could configure for only 10 updates, but I want it all in 1 PR or 10 PRs. One of the killer features is its ability to configure the age of a package before updating. I’ve found this to be indispensable in not updating packages before they are battle-tested by other consumers first.
The drawback is that it is a command-line tool (or a dotnet global tool). This makes it rather difficult to automate at an organizational level. Sure there are ways to do it. You can include the execution of the global tool in your project during a CI build. In an attempt to build something more agnostic, so I didn’t have to build it into every build pipeline, I put together a script, a YAML manifest and a single nightly build that would loop through and execute on each repository I had configured. This allowed the team to submit a simple PR to add new repositories without much additional work.
NuKeeper and this YAML configuration worked well for a short period of time. Its infrastructure necessary to configure internal private NuGet feeds was sloppy. Some weird exceptions ended up being problematic over time for projects with Pull Requests already submitted. It broke down after a while from the enterprise configuration and execution perspective.
During my NuKeeper analysis, I briefly looked into Dependabot as well. At the time it was a paid product (unless you were working on a public or open source project) and not something I got too far with. It looked very promising with a SaaS type of an approach, while also supporting a very wide variety of package managers. In May 2019, Dependabot announced it is joining GitHub and its public preview is now available for free or private and/or organizational usage. This changes things up quite a bit.
Over the past few months, we have been able to fully deprecate our usage of NuKeeper in favor of the Dependabot preview. This unlocked the same features we were using for .NET now across all our package managers. Thus far with its usage, my feedback is very favorable:
- package_manager: "dotnet:nuget"
- package_manager: "docker"
It is also worth noting that GitHub has had their “Security Vulnerability” scanning feature available in their repositories for some time now. This feature is now integrated with Dependabot and you can explicitly receive Pull Requests from Dependabot for packages with identified security threats only if you wish. This is exciting to see the intersection of the GitHub features in this way. As an aside, I expect that in the future we’ll begin to see some very interesting compositions between Microsoft, GitHub Actions, Dependabot and NPM with GitHub.
Dependabot certainly scratches the itch that I’ve been trying to get at for awhile, but there are always some caveats. Consider the following thoughts and suggestions for efficiency as you get started:
I find myself still wanting some of the configuration features available in NuKeeper. The ability to configure the age of the packages was incredible. Under GitHub, I’m hoping we’ll see some of these more advanced features make their way into Dependabot, perhaps when it emerges from “preview”?
Dependabot is still in preview. This is always a concern when analyzing and rolling out tooling to your organization and/or enterprise. That being said, I can only see a bright future for the integration of Dependabot with GitHub, and we have chosen to accept the risk and dive in based on all the problems it can solve (it also feels low risk in the surface area if it is a flop). The Dependabot GitHub app does allow you to “enable for your organization” as a whole. This allows your developers in the organization to simply add that config.yml and begin receiving Pull Requests (no other configuration needed).
There are of course many advantages to incrementally staying up to date with the latest package dependencies that we have mentioned. But another internal advantage that is a large driver in pushing Dependabot is the integration with internal or private package feeds as well. This gives your organization the ability to push updates when internal package dependencies are updated as well. We share a number of libraries across teams, products, and projects. It is essential that when we make a bug fix, security patch or even just add a new feature that the library is pushed out to all consumers automatically and notifies them of exactly what has changed (Dependabot includes a list of GIT Commits in that release from the repository automatically in each Pull Request). This allows our teams to move fast with less friction on shared libraries.