/auditing-tls-certificate-transparency-logs

name: auditing-tls-certificate-transparency-logs
description: 'Monitors Certificate Transparency (CT) logs to detect unauthorized certificate issuance, discover subdomains
  via CT data, and alert on suspicious certificate activity for owned domains. Uses the crt.sh API and direct CT log querying
  based on RFC 6962 to build continuous monitoring pipelines that catch rogue certificates, track CA behavior, and map the
  external attack surface. Activates for requests involving certificate transparency monitoring, CT log auditing, subdomain
  discovery via certificates, or certificate issuance alerting.

  '
domain: cybersecurity
subdomain: threat-intelligence
tags:
- certificate-transparency
- CT-logs
- crt-sh
- subdomain-discovery
- TLS-monitoring
- RFC-6962
version: 1.0.0
author: mukul975
license: Apache-2.0
nist_csf:
- ID.RA-01
- ID.RA-05
- DE.CM-01
- DE.AE-02

Auditing TLS Certificate Transparency Logs

When to Use

• Monitoring owned domains for unauthorized or unexpected certificate issuance by unknown Certificate Authorities
• Discovering subdomains and hidden services through certificates logged in public CT logs
• Detecting phishing infrastructure that uses look-alike domain certificates (typosquatting, homograph attacks)
• Auditing Certificate Authority compliance by verifying all issued certificates appear in CT logs as required by browser policies
• Building continuous certificate monitoring into a security operations pipeline with alerting for new issuances

**Do not use** for attacking or disrupting Certificate Authorities, for scraping CT logs in violation of rate limits or terms of service, or as the sole method of subdomain enumeration without corroborating results through DNS verification.

Prerequisites

• Python 3.10+ with `requests`, `cryptography`, and `pyOpenSSL` libraries installed
• Network access to crt.sh (HTTPS) and public CT log servers
• A list of domains to monitor (owned domains, brand variations, typosquat candidates)
• SMTP credentials or webhook URL for alerting on new certificate discoveries
• Basic understanding of X.509 certificate structure and TLS certificate chain validation

Workflow

Step 1: Domain Inventory and Baseline

Build the initial certificate inventory for monitored domains:

• **Define monitoring scope**: List all owned root domains, registered brand names, and known subsidiaries. Include wildcard patterns (`%.example.com`) for comprehensive subdomain coverage.
• **Query crt.sh for historical certificates**: Use the crt.sh JSON API to retrieve all known certificates for each domain. The API endpoint `https://crt.sh/?q=%.example.com&output=json` returns certificates matching the wildcard pattern with fields including `issuer_ca_id`, `issuer_name`, `common_name`, `name_value`, `not_before`, `not_after`, and `serial_number`.
• **Build baseline database**: Store the initial certificate set in a local SQLite database with columns for certificate ID, domain, issuer, validity dates, SANs (Subject Alternative Names), and first-seen timestamp. This baseline prevents alerting on already-known certificates.
• **Identify authorized CAs**: From the baseline, extract the set of Certificate Authorities that have legitimately issued certificates for your domains. Any future issuance from a CA not in this set triggers a high-priority alert.
• **Map subdomains**: Extract all unique subdomains from the `name_value` field across all certificates to build an initial subdomain inventory.

Step 2: Continuous CT Log Monitoring

Set up ongoing monitoring for new certificate issuances:

• **Poll crt.sh periodically**: Query the crt.sh API at regular intervals (every 15-60 minutes) for new certificates. Use the `exclude=expired` parameter to focus on currently valid certificates. Compare results against the baseline database to identify new entries.
• **Parse certificate details**: For each new certificate, extract the full SAN list, issuer chain, validity period, key type and size, CT log SCT (Signed Certificate Timestamp) details, and certificate fingerprint (SHA-256).
• **Detect precertificates**: CT logs include precertificates (poisoned certificates submitted before final issuance). Track these as early warnings since they indicate a certificate is about to be issued but may not yet be active.
• **Monitor CT log Signed Tree Heads (STH)**: For advanced monitoring, query CT log servers directly to fetch the latest STH and verify consistency proofs between consecutive tree heads. An inconsistency indicates log misbehavior (split-view attack).
• **Rate limiting awareness**: Respect crt.sh rate limits by spacing queries and caching responses. Implement exponential backoff on HTTP 429 responses. For high-volume monitoring, consider querying the crt.sh PostgreSQL interface directly at `crt.sh:5432`.
• **Atom/RSS feed alternative**: Subscribe to crt.sh's Atom feed for lighter-weight monitoring: `https://crt.sh/atom?q=%25.example.com` provides real-time notification of new log entries.

Step 3: Subdomain Discovery via CT Data

Extract and validate subdomains found in certificate transparency data:

• **Wildcard expansion**: Certificates with wildcard SANs (`*.dev.example.com`) reveal the existence of subdomains that may not be in DNS zone files. Record the parent domain as a target for further enumeration.
• **Historical certificate mining**: Query crt.sh without the `exclude=expired` parameter to find subdomains from expired certificates that may still resolve in DNS. These represent historical infrastructure that could be vulnerable to subdomain takeover.
• **DNS validation**: For each discovered subdomain, perform DNS resolution (A, AAAA, CNAME records) to determine if the subdomain is currently active. Cross-reference with known IP ranges to identify shadow IT or unauthorized services.
• **Typosquat detection**: Generate permutations of the monitored domain (bitsquatting, homograph, insertion, omission, transposition, keyboard-adjace