Discovering Dependabot many years ago was one of those moments of incredible excitement. It was not the same kind of excitement as when you finished creating and publishing that new feature to production, or even the same as when you got some amazing feedback about how your software helped save someone a ton of time… or maybe even changed their lives. This type of excitement is what you get when you realize someone else’s code is going to either change the way you develop and/or increase your quality of life. It is the epiphany that you get when you realize that this is the way it should have been all along… but for some reason wasn’t, and you could never place your finger on it. Dependabot was a breath of fresh air, in a world of DevSecOps that felt like my responsibility and requirements were ever-increasing. Dependabot can make at least a sliver of that pain go away.
This excitement grew even larger with GitHub’s acquisition of the Dependabot product and its now native integration and migration away from Dependabot-Preview to the Dependabot that we know today. GitHub took a good deal longer than I think most of us expected to make the transition, and for a good chunk of time, there were key features missing that should have been part of the MVP.
With GitHub native integration of Dependabot, SPS Commerce very quickly took the first steps to offer standardized integration, examples, and organizational secrets for interacting with our private registries. From a Developer Experience perspective, we wanted all our internal teams and engineers to take advantage of this simple yet effective security best practice. Unfortunately, like anything at scale, it was not that simple. We continually found that Dependabot adoption would get hung up on integrating with the last bits of our teams’ workflows. Consistent adoption struggles:
GitHub released more organizational governance controls. This enabled security teams to easily and broadly turn on Dependabot Security Alerts across the organization (backed by the Dependency Graph feature), also providing real-time SBOMs. Controls are available to turn on Dependabot Security Updates across the organization as well, though we found out very quickly the effect of this seemingly innocent addition had a large negative impact. While security alerts simply inform you about critical security issues, security updates create Pull Requests. Enabling across the organization in a single button click created Pull Requests at lightning speed across over 4,000 repositories, with each Pull Request potentially firing CI/CD integration requests and queued jobs for validation. While our centralized CI/CD platform was chugging through the queue we reached numerous limits and throttling, one in particular, in the CI/CD platform getting throttled for GitHub API requests for source code.
The adoption of Dependabot Security Alerts was global, but actual Security Updates are hit or miss without solving some of the notable issues above first.
In my blog post a few years ago on migrating to Dependabot Native, I talked about the need for Grouped Updates. This single feature can alone have a substantial impact on several of the problems we have discussed so far. Given the impact of this feature, we evaluated the possible migration to Renovate as an alternative that already has support for Grouped Updates (and plenty of other features). Ultimately we decided to wait for the feature from GitHub in an attempt to curb our tool sprawl and keep the ecosystem and scope tight. GitHub had originally indicated in our chats that they expected a beta of Grouped Updates would be available by the end of the year 2021, which unfortunately was not the case as it was substantially delayed for a good 18 months past that, but is finally available and in GA!
With grouped updates available, custom patterns can now enable any grouping of wildcarded package names that you might need, with built-in support for production and development dependency ecosystems. Grouped updates have a major impact on just about every issue area noted above:
Issues around package maturity have not been solved, but I have had a lot of success in moving to monthly update intervals, which seems reasonable for a regular update cadence of the dependencies. With Grouped Updates, I may feel more comfortable moving to weekly update intervals.
Here is an example of a typical .NET6 API
- package-ecosystem: nuget
- dependency-name: "Microsoft.Extensions.*"
- dependency-name: "Microsoft.AspNetCore.*"
- dependency-name: "System.Text.Json"
This creates 4 standard pull request types that can be opened, with logical grouping in each. Open Telemetry (“otel”) and AWS packages are numerous and are helpful to position as a single Pull Request. But most importantly all Microsoft Extensions and .NET system references will be upgraded together. In my example, the intent is to only update framework packages for “minor” or “patch” updates as explicitly not upgrading beyond the existing LTS Support is desirable (see Update Types). All of these complexities can now be codified generically in the Dependabot configuration. Nice!
With the initial governance controls now available with grouped updates, I am hoping that Dependabot Version Updates is now more appropriately enterprise-ready than prior initiatives. We will continue to provide training, and recommendations and encourage the adoption of Dependabot at SPS Commerce given the impact on both security and Developer Experience, using the launch of Grouped Updates as a change event to re-engage in the initiative. I am hoping this is the first of many great new features and updates coming to Dependabot.
I believe also that Dependabot can and should play a major role in an organization’s Inner Source strategy. With the distribution of internal shared libraries, it is essential to keep new versions flowing and reduce the support of multiple active versions in a timely manner. Communication of updates is difficult but can be very simple and effective using Dependabot (for release notes as well). More to come in the future about SPS Commerce and its growing Inner Source strategy and best practices.
What other features and enhancements would you like to see as part of Dependabot?