GitHub Switches to Faster, Cheaper AI Model for Secret Detection

GitHub has shifted from a large‑language‑model pipeline to a lightweight classifier built on ModernBERT, a move that cuts the median time for scans after a push by 86 % and reduces the cost of serving the model by more than 98 % compared with the previous system. The change also lets GitHub extend generic secret detection to its push‑protection stage, giving developers a first line of defence against credential leaks before code enters the repository. By speeding up scans, the new pipeline keeps the platform responsive for millions of users without compromising security.

The secret‑scanning process begins with regular‑expression rules that flag strings that look like passwords, tokens or keys. Those candidates are then fed into a fine‑tuned ModernBERT model that uses the file path, variable names, and surrounding code or text to score each string. When the score surpasses a threshold, the candidate triggers a post‑push alert or blocks a push, while likely false positives are filtered out. This combination of pattern matching and contextual analysis enables detection of generic secrets that might otherwise be missed by rule‑based systems.

GitHub built training data from real repository examples identified by regex, product feedback, and synthetic cases that cover uncommon or ambiguous secrets. Human reviewers validated and annotated the examples, and the team grouped near‑duplicate rows to avoid leakage between training and evaluation sets. In offline tests on public traffic, the new pipeline achieved 13 % higher precision and 19 % higher recall than the older LLM approach, and an A/B experiment showed an 11.5 % rise in alert acceptance weighted by repository owner. GitHub plans to use ongoing production feedback to fine‑tune the model and is exploring architectures that could combine detection and extraction in a single step.