Your AI Gateway May Be Open to Anyone Who Tries 'sk-1234'
Your AI Gateway May Be Open to Anyone Who Tries 'sk-1234'
Most enterprise security teams have spent years hardening identity. Conditional Access policies, Intune compliance gates, phishing-resistant MFA — the authentication stack is tighter than it has ever been. Then a developer commits an OpenAI API key to a private GitHub repo, and none of that work matters anymore.
Why API Keys Break the Identity Perimeter
The Microsoft identity perimeter is built on a clear assumption: every access request flows through Entra ID, where it can be evaluated against policy. Conditional Access inspects the user, the device, the network location, the application, and the risk signal before granting a token. Intune compliance status feeds into that evaluation. Named locations block unexpected geographies. Sign-in risk policies catch anomalous behavior.
API keys exist entirely outside this model.
When a script or application authenticates to an AI service — Azure OpenAI, the OpenAI API, Anthropic, or any third-party AI gateway — using a static key, it presents that key directly to the service endpoint. There is no Entra token exchange. There is no Conditional Access evaluation. The service sees a valid key and responds. The identity plane never knew the request happened.
The structural problem: API keys are credentials without identity. They authenticate a string, not a person or a device. Every security control built on identity — MFA, Conditional Access, Privileged Identity Management — is irrelevant once a key is in play.
This is not a new problem in enterprise IT. Database connection strings, SMTP relay credentials, and webhook secrets have always carried this risk. What has changed is scale. The rapid adoption of AI APIs across development teams, automation pipelines, and even end-user tools has dramatically expanded the surface area. Keys are being generated faster than governance frameworks can track them.
The Exposure Vectors That Actually Matter
Understanding where keys leak is more useful than a generic warning about "poor key hygiene." In practice, the exposure vectors that cause real incidents fall into a short list.
Source code repositories remain the most common vector. Developers working quickly commit .env files, configuration files, or inline key assignments. Even in private repositories, this creates risk — former employees retain access, repository permissions drift, and accidental public exposure happens. GitHub's secret scanning feature catches some patterns, but it is opt-in and does not cover every key format.
Build and CI/CD pipeline logs are frequently overlooked. When a key is passed as an environment variable and a verbose logging step prints environment state, the key appears in plain text in the build log. Many CI/CD platforms retain logs for extended periods, and log access controls are often less rigorous than code access controls.
Container images pushed to registries sometimes contain baked-in keys. A developer who hardcodes a key during local development and then builds a container image without cleaning the configuration has effectively published that key to every environment where the image runs.
Collaboration tools — Teams, Slack, email — are a consistent source of accidental exposure. A developer sharing a key for a colleague to test with creates a persistent record in a channel that may be accessible to dozens of people.
Application error logs and crash dumps can expose keys when an application logs its configuration state during an exception. If those logs flow to a centralized logging platform with broad read access, the key is effectively shared.
What Microsoft's Native Tooling Does and Does Not Cover
It is worth being precise about where the Microsoft security stack helps and where it does not.
Microsoft Defender for Cloud Apps can detect OAuth token anomalies and flag unusual API activity for cloud applications that are connected through the app connector framework. It does not natively inspect traffic to third-party AI APIs or detect static key usage patterns in code or logs.
Microsoft Purview Data Loss Prevention can be configured to detect patterns that resemble API keys in documents, emails, and Teams messages using custom sensitive information types. This requires deliberate configuration — it does not happen automatically. A custom regex pattern targeting common AI API key formats (sk-[a-zA-Z0-9]{48} for OpenAI, for example) can be added as a custom sensitive information type and applied to DLP policies across Exchange, SharePoint, Teams, and endpoint.
Microsoft Defender for DevOps (part of Defender for Cloud) integrates with GitHub, Azure DevOps, and GitLab to surface secrets exposed in repositories. This is the most directly relevant native control, but it requires the integration to be configured and the repositories to be onboarded. It does not retroactively scan historical commits without additional tooling.
Microsoft Sentinel can ingest logs from AI service providers if those providers support log export. Azure OpenAI logs can be routed to a Log Analytics workspace, enabling KQL-based detection rules. Third-party AI APIs require custom data connectors or HTTP data collector API ingestion.
The honest assessment: Microsoft's tooling covers Azure OpenAI reasonably well if you configure it intentionally. It covers third-party AI APIs poorly without significant custom work.
Detection: PowerShell and KQL in Practice
Detection has to happen at two layers — the code and configuration layer, and the runtime traffic layer.
For the code layer, a PowerShell scan across local development environments or file shares can surface hardcoded keys before they reach a repository. The following script targets common AI API key patterns across multiple file types:
$scanPaths = @("C:\Dev", "C:\Projects", "C:\Scripts")
$keyPatterns = @(
"sk-[a-zA-Z0-9]{20,}", # OpenAI format
"sk-ant-[a-zA-Z0-9\-]{30,}", # Anthropic format
"AIza[0-9A-Za-z\-_]{35}", # Google AI format
"Bearer\s+[a-zA-Z0-9\-_\.]{40,}" # Generic bearer token pattern
)
$results = @()
foreach ($path in $scanPaths) {
Get-ChildItem -Path $path -Recurse -Include *.py,*.js,*.ts,*.cs,*.json,*.env,*.yaml,*.yml,*.config -ErrorAction SilentlyContinue | ForEach-Object {
$file = $_
$content = Get-Content -Path $file.FullName -Raw -ErrorAction SilentlyContinue
foreach ($pattern in $keyPatterns) {
if ($content -match $pattern) {
$results += [PSCustomObject]@{
File = $file.FullName
Pattern = $pattern
Modified = $file.LastWriteTime
}
}
}
}
}
if ($results.Count -gt 0) {
$results | Format-Table -AutoSize
$results | Export-Csv -Path "C:\Temp\APIKeyExposureReport.csv" -NoTypeInformation
Write-Warning "$($results.Count) potential API key exposure(s) detected. Review the report immediately."
} else {
Write-Output "No hardcoded API key patterns detected in scanned paths."
}Run this script as a scheduled task on developer workstations managed through Intune, using a PowerShell script deployment policy. Output the CSV to a monitored share or ingest it into Log Analytics using the HTTP Data Collector API for centralized visibility.
For runtime detection in Azure OpenAI environments, enable diagnostic logging on the Azure OpenAI resource and route logs to a Log Analytics workspace. Then apply a KQL detection rule in Microsoft Sentinel:
// Detect Azure OpenAI requests from unexpected source IPs or high-volume anonymous patterns
AzureDiagnostics
| where ResourceType == "OPENAI"
| where OperationName == "ChatCompletions_Create" or OperationName == "Completions_Create"
| summarize RequestCount = count(), UniqueCallerIPs = dcount(CallerIPAddress) by bin(TimeGenerated, 1h), Resource
| where RequestCount > 500 or UniqueCallerIPs > 10
| project TimeGenerated, Resource, RequestCount, UniqueCallerIPs
| order by RequestCount descHigh request volume from multiple source IPs against a single Azure OpenAI resource is a strong indicator that a key has been shared or leaked. A legitimate application calling from a fixed set of infrastructure IPs will not produce this pattern.
For detecting API key patterns in Microsoft Purview audit logs — for example, keys pasted into Teams or SharePoint — configure a custom sensitive information type in the Purview compliance portal using the regex patterns above, then apply a DLP policy with an audit-only action initially to baseline the volume before moving to block or notify.
Governance Architecture for AI API Credentials
Technical detection is necessary but not sufficient. The governance model has to change to treat AI API keys as privileged credentials, not developer convenience tokens.
Azure Key Vault is the correct storage location for any API key used in production workloads. Key Vault provides encryption at rest, access control through Entra ID RBAC, audit logging of every access event, and support for key rotation without application redeployment when combined with Key Vault references in Azure App Service and Azure Functions. Applications retrieve secrets at runtime using managed identities — no key ever appears in code or configuration files.
Managed identities should be the default for any Azure-hosted workload calling Azure OpenAI. A managed identity authenticates through Entra ID, which means the call is subject to Conditional Access, appears in sign-in logs, and carries a verifiable identity. This is the architectural pattern that restores the identity perimeter for AI service calls.
Key rotation policy should be defined and enforced. For Azure OpenAI, keys can be rotated through the Azure portal or via the Azure CLI without service interruption if applications are retrieving keys from Key Vault rather than hardcoding them. A 90-day rotation cycle is a reasonable baseline; environments subject to PCI DSS or FedRAMP have stricter requirements.
Inventory and ownership are the governance gaps that cause the most operational pain. When a key is compromised and must be rotated immediately, the question "what applications use this key?" should have an immediate answer. If it does not, the incident response timeline extends significantly. Maintain a key inventory in a configuration management database or a dedicated secrets management platform, with each key mapped to an owner, a service, and a rotation date.
For compliance frameworks including SOC 2 Type II and ISO 27001, the absence of a key inventory and rotation policy is a finding. Auditors increasingly ask specifically about AI service credentials as AI adoption has accelerated across enterprise environments.
Recommendations That Change the Risk Profile
The following steps address the actual attack surface rather than the theoretical one.
Audit existing key usage before anything else. Run the PowerShell scan above across development environments, CI/CD pipeline configurations, and any shared file locations. Identify every key that exists, where it is stored, and what service it authenticates. This baseline is the starting point for remediation.
Migrate production workloads to managed identities. For any Azure-hosted application calling Azure OpenAI, replace static key authentication with a system-assigned or user-assigned managed identity. Assign the managed identity the Cognitive Services OpenAI User role on the Azure OpenAI resource. Remove the static key from the application configuration entirely.
Deploy Defender for DevOps and onboard all GitHub and Azure DevOps repositories. Enable secret scanning and configure push protection to block commits containing detected secrets before they reach the repository. This is a preventive control, not just a detective one.
Configure Purview DLP with custom sensitive information types targeting AI API key formats. Apply the policy in audit mode for 30 days to establish a baseline, then move to notify-and-block for high-confidence matches in Teams, Exchange, and SharePoint.
Establish a key compromise runbook. When a key is suspected to be compromised, the response steps should be documented and tested: identify all applications using the key, rotate the key in the source service, update Key Vault with the new value, verify application continuity, and review access logs for the period of potential exposure. A runbook that has never been tested will fail under incident pressure.
Final Thoughts
The identity controls that enterprise security teams have built over the last several years are genuinely strong. The problem is that API keys route around them entirely. Every AI service key that exists outside a secrets management system, without rotation policy, without an inventory entry, and without audit logging is a credential that operates in a blind spot.
The fix is not complicated, but it requires treating AI API keys with the same discipline applied to privileged account credentials. Managed identities eliminate the problem for Azure-hosted workloads. Key Vault solves the storage problem. Defender for DevOps and Purview DLP address the detection gap. The governance work — inventory, ownership, rotation policy, incident runbook — is what makes the technical controls durable.
The organizations that get this right are the ones that audit first, migrate to managed identities where possible, and build detection before an incident forces the conversation.
---
