Secrets embedded in source code pose a risk to developers and the organizations they work in. Secrets can be used to take over both user and service accounts, which can lead to sensitive data exposure, operational risks, and financial or reputational damage.
There are many commercial and open source projects available to detect hardcoded secrets, but mitigating — by removing, obfuscating, or rotating — exposed secrets is extremely hard. For example, rotating a secret tied to a service in your cloud environment can cause a denial of service across your application if that service is consumed by other code where the secret is not detected. This may be the case even if the secret is stored securely in a secrets management service.
The risk of having secrets in a source code repository is that the secret is immediately exposed to any other users who clone this repository. All historical secrets can be detected by scanning the files or the commits in the git history of the repository.
Therefore, mitigating secrets before they are exposed in the files or git history is a more desirable approach. Pipelineless security enforces a “zero new hardcoded secrets” policy by keeping secrets out of the source code repositories.
Where Are Hardcoded Secrets Traditionally Detected?
A typical code delivery flow begins with a developer cloning or downloading the latest version of the code repository and making changes to the local repository. Pushing the code changes back to the remote repository often kicks off a series of checks and scanners — for example, GitHub supports Checks API, while Azure DevOps supports service webhooks. Before the new features (or fixes) can be deployed to production, the developer’s code branch needs to be merged into the main branch, which typically triggers another round of checks and build/CI pipelines to ensure that the application will still work properly after the merge. For example, container builds can occur at this time to speed up the pipeline after the code is merged.
Secret-scanning checks can be implemented in several places in the developer workflow: IDE plugins, pre-commit hooks, pre-receive hooks, webhooks to scanner, and CI/CD pipeline. Each method has pros and cons.
IDE plugins. Secrets can be scanned as the developers write code. It is an effective approach to help avoid committed secrets into the git history, but due to the various IDE tools and versions within engineering organizations, it is hard to ensure that 100% of developers will use it. This approach lacks an enforcement capability, which means developers can bypass this control — a serious drawback to any client-side security control.
Pre-commit hooks. Automated scripts run before code is committed into the local git history. This control has the same pros and cons as IDE plugins. Additionally, commits resulting from suggested code changes on a pull request can end up unscanned if they get accepted by the developer.
Pre-receive hooks. These are like pre-commit hooks, but they run on the source code management (SCM) side. This means that 100% coverage can be reached. However, they require administrative access to the underlying operating system, which cloud-based offerings such as GitHub cannot provide.
Webhooks to scanner. This is an effective approach, as it can provide 100% coverage as well. While webhooks cannot prevent push events from successful execution, they are delivered quickly and can be easily deployed on both cloud and on-premises deployments, which makes them very popular. Note that the results of the GitHub Checks API are visible to all users with access to the repository — which means that if a secret is exposed, others will know.
Pipeline. Implementing secrets scanning across 100% of the repositories is hard because pipelines typically require code changes. Additionally, it takes time to get a runner to kick off and execute a job, which makes it less effective when it comes to immediate secret mitigation. As is the case with webhooks, the results of the pipelines are visible to all users with access to the repository.
Where ‘Zero New Hardcoded Secrets’ Comes In
The concept of zero new secrets is fairly simple. A service that enforces such a policy needs to be built with the following in mind:
- Subscribe your secret mitigation service to all important events across all repositories, e.g. subscribe to code push, pull requests, and audit events.
- Kick off out-of-band automated git-based guardrails, such as resetting a branch to a pre-pushed state.
- Provide isolated feedback to the code pusher, at minimum. Ensure that the new secret finding is not presented to unnecessary users.
This is also known as pipelineless security. In a pipelineless implementation, a series of checks and services handles secrets detection right as the code is being pushed to the remote repository. When the developer pushes code from the local branch to the remote repository, a webhook is sent to the pipelineless security service, which runs the secret detection script. If no secret is found, the code is pushed normally.
However, if a secret is detected, the service resets the branch to the pre-push state. This way, the commits containing the secrets will not be accessible to adversaries poking around the repository’s commit history. The developer is then notified privately and given instructions on how to back up the changes and fetch the code so that it can be fixed — both to minimize the risk of hardcoded secret exposure and to prevent any stigma from attaching to the developer.
A zero new hardcoded secrets policy, enabled by a pipelineless approach, can be effective in preventing the exposure of new secrets while developers work to resolve the backlog of historical secrets. Such a policy offers 100% coverage, the ability to mitigate newly pushed secrets in seconds, and direct feedback to the developer to act without exposing code to unnecessary parties.
