Accidentally Committed Secrets? A Simple Git Fix Is Not Enough!
It happens to the best of us. A moment of distraction, and suddenly we've committed a .env file, an API key, or another secrets file to our Git repository. If you think simply removing the file and committing again will fix the issue, think again. The file will still exist in your Git commit history, posing a security risk. In this article, we’ll explore how to effectively remove secrets from Git history and mitigate potential security risks. The Problem When we commit a file to a Git repository and then remove it in a subsequent commit, the file isn't actually deleted from the repository's history. Git is designed to maintain a complete historical record, which means anyone with access to our repository can: Browse through previous commits View the contents of those commits, including our secrets Extract sensitive information with simple Git commands # For example, someone could do this to see our secrets: git checkout cat path/to/secrets/file The Solution Scenario 1: Haven't Pushed to Remote Yet If the commit with secrets is still only in our local repository. This is the easiest scenario to fix. : Use git reset to undo the commit Remove the sensitive data Commit again with the sanitized files # Undo the last commit, but keep the changes in our working directory git reset --soft HEAD~1 # Alternatively, if we want to discard the changes completely git reset --hard HEAD~1 Scenario 2: Already Pushed to Remote To truly remove sensitive information, we need to rewrite our Git history to completely eliminate the file from all commits. Here's how to do it effectively: Step 1: Identify the Secret Files First, clearly identify which files contain sensitive information that needs to be removed. Step 2: Remove the Files from Git History 1) Using git filter-branch git filter-branch --force --index-filter "git rm --cached --ignore-unmatch PATH_TO_SECRET_FILE" --prune-empty --tag-name-filter cat ----all Replace PATH_TO_SECRET_FILE with the path to file containing secrets. For replacing specific strings (like API keys) while keeping the file: git filter-branch --tree-filter "find . -type f -name '*.config' -exec sed -i 's/YOUR_API_KEY/PLACEHOLDER_KEY/g' {} \;" HEAD 2) Using filter-repo git-filter-repo isn't built-in to git itself First, install git-filter-repo: # For Python users pip install git-filter-repo # For macOS brew install git-filter-repo Then run: git filter-repo --path path/to/secrets/file --invert-paths for more info regarding git-filter-repo refer this Step 3: Push those Changes After cleaning history, we need to force push the changes: git push origin --force --all CAUTION: Force Pushing Is Irreversible ⚠️ Additional Recommendations Revoke any exposed API keys or tokens Add sensitive files to .gitignore: Ensure secret files are never tracked. Use Environment Variables: Store secrets in environment variables instead of committing them. Use Git Hooks: Automate pre-commit hooks to prevent secret files from being added. Use GitHub’s Secret Scanning If you're using Github or use Secret push protection for Gitlab. Note: Remember that if your secret was exposed for any period of time, even briefly, you should consider it compromised and rotate it immediately.
It happens to the best of us. A moment of distraction, and suddenly we've committed a .env file, an API key, or another secrets file to our Git repository. If you think simply removing the file and committing again will fix the issue, think again. The file will still exist in your Git commit history, posing a security risk.
In this article, we’ll explore how to effectively remove secrets from Git history and mitigate potential security risks.
The Problem
When we commit a file to a Git repository and then remove it in a subsequent commit, the file isn't actually deleted from the repository's history. Git is designed to maintain a complete historical record, which means anyone with access to our repository can:
- Browse through previous commits
- View the contents of those commits, including our secrets
- Extract sensitive information with simple Git commands
# For example, someone could do this to see our secrets:
git checkout
cat path/to/secrets/file
The Solution
Scenario 1: Haven't Pushed to Remote Yet
If the commit with secrets is still only in our local repository. This is the easiest scenario to fix. :
- Use
git resetto undo the commit - Remove the sensitive data
- Commit again with the sanitized files
# Undo the last commit, but keep the changes in our working directory
git reset --soft HEAD~1
# Alternatively, if we want to discard the changes completely
git reset --hard HEAD~1
Scenario 2: Already Pushed to Remote
To truly remove sensitive information, we need to rewrite our Git history to completely eliminate the file from all commits. Here's how to do it effectively:
Step 1: Identify the Secret Files
First, clearly identify which files contain sensitive information that needs to be removed.
Step 2: Remove the Files from Git History
1) Using git filter-branch
git filter-branch --force --index-filter "git rm --cached --ignore-unmatch PATH_TO_SECRET_FILE" --prune-empty --tag-name-filter cat ----all
Replace PATH_TO_SECRET_FILE with the path to file containing secrets.
For replacing specific strings (like API keys) while keeping the file:
git filter-branch --tree-filter "find . -type f -name '*.config' -exec sed -i 's/YOUR_API_KEY/PLACEHOLDER_KEY/g' {} \;" HEAD
2) Using filter-repo
git-filter-repo isn't built-in to git itself
First, install git-filter-repo:
# For Python users
pip install git-filter-repo
# For macOS
brew install git-filter-repo
Then run:
git filter-repo --path path/to/secrets/file --invert-paths
for more info regarding git-filter-repo refer this
Step 3: Push those Changes
After cleaning history, we need to force push the changes:
git push origin --force --all
CAUTION: Force Pushing Is Irreversible ⚠️
Additional Recommendations
- Revoke any exposed API keys or tokens
- Add sensitive files to
.gitignore: Ensure secret files are never tracked. - Use Environment Variables: Store secrets in environment variables instead of committing them.
- Use Git Hooks: Automate pre-commit hooks to prevent secret files from being added.
- Use GitHub’s Secret Scanning If you're using Github or use Secret push protection for Gitlab.
Note: Remember that if your secret was exposed for any period of time, even briefly, you should consider it compromised and rotate it immediately.
