During an external penetration test, and especially if it is a black-box engagement, one of the most important steps is the discovery of subdomains used by the target company.
Subdomain Enumeration is a process of finding sub-domains of one or more root domains. A good subdomain enumeration will help you find those “hidden” subdomains where there are possibilities in finding hidden or “forgotten” services which may lead to the uncovering of critical vulnerabilities.
So, how can you discover subdomains during a penetration testing engagement?
There are several tools and methods which can be utilized to discover subdomains on a given target domain.
Many tools will return duplicate results of course. You should exhaust all available resources to retrieve as many subdomains as possible to expand your attack surface for the target company.
Let’s look into some resources where you can find subdomains both passively and actively.
–> The links to the resources are on titles and/or inline text.
1 – Google Search
2 – Google Analytics
3 – NMapper
4 – CRT.SH
5 – SecurityTrails
6 – Pentest-Tools
7 – Spyse
8 – DNSDumpster
9 – Censys
10 – BinaryEdge
11 – SubdomainFinder
12 – Shodan
13 – Scantrics.io
14 – Amass
15 – DNSrecon
16 – Shodomain
17 – Sublist3r
18 – SubBrute
19 – Knock
20 – theHarvester
21 – Sudomy
22 – Subfinder
Google hacking, or otherwise google dorking, can be used to find subdomains of any domain name.
You can perform subdomain enumeration via Google Analytics ID with a tool called AnalyticsRelationships. After you download the version you prefer (there is python, go and docker variants of the tool) you may run a search like the below:
./analyticsrelationships -url https://www.example.com
Nmapper is an online subdomain finder using Sublist3r DNScan Anubis Amass Lepus Findomain Censys.
Find subdomains with crt.sh through the certificate fingerprint used on the websites.
One of the largest repositories of historical DNS data. Create an account for further access to the search data.
A few free scans can be provided by PenTest-Tools.
Spyse [Update: Currently Unavailable]
Search with Spyce and get subdomains, their DNS records, locations, HTTP status codes, Site titles, ASNs, and technologies used.
DNSDumpster is a free domain research tool to discover subdomains. It returns more details like the IP block owners, GeoIP information, hosts sharing the same IP address, technologies used by the targets, active Nmap scans online, and more.
Censys is an amazing tool. You can benefit from it differently, like getting information about websites or IP addresses. Although it is not intentionally meant to retrieve subdomain information, it can still be more useful than many other free services.
Another tool is BinaryEdge. Although it is not completely free, it is a powerful tool to find subdomains. It offers 250 queries a month for the free version.
Subdomainfinder is quite a good subdomain search engine. It also provides information on whether the host is behind Cloudflare.
The infamous Shodan search engine! Use its search capabilities and find subdomains with filtered queries like: ssl:example.*200 or with https://www.shodan.io/domain/example.com
There are several command line tools utilizing Shodan’s API which you can also use to search for subdomains. I will reference them later in the article.
Scantrics.io [Update: Currently Unavailable]
This tool utilizes several methods to retrieve subdomains for your target domain, with quite a few details like IPs, Whois info, location, OS and server information, and page title/banner.
The OWASP Amass Project performs network mapping of attack surfaces and external asset discovery using open source information gathering and active reconnaissance techniques.
amass -d exmaple.com
More examples of the usage of amass can be found here.
DNSRecon is a python script that provides the ability to perform:
This script provides the ability to perform:
- Check all NS Records for Zone Transfers.
- Enumerate General DNS Records for a given Domain (MX, SOA, NS, A, AAAA, SPF and TXT).
- Perform common SRV Record Enumeration.
- Top Level Domain (TLD) Expansion.
- Check for Wildcard Resolution.
- Brute Force subdomain and host A and AAAA records given a domain and a wordlist.
- Perform a PTR Record lookup for a given IP Range or CIDR.
- Check a DNS Server Cached records for A, AAAA and CNAME Records provided a list of host records in a text file to check.
dnsrecon -a -d example.com
Shodomain is a python script able to grab and print subdomains from Shodan API.
python shodomain.py API-KEY example.com
Sublist3r is a python tool designed to enumerate subdomains of websites using OSINT
python sublist3r.py -d example.com
SubBrute uses open resolvers as a kind of proxy to circumvent DNS rate-limiting
Knockpy is a python3 tool designed to quickly enumerate a target domain’s subdomains through a dictionary attack.
python3 knockpy.py example.com
The Harvester tool gathers emails, names, subdomains, IPs and URLs using multiple public data sources.
theHarvester -d example.com -l 500 -b google
Sudomy is a fast subdomain enumeration and analyzer. It can utilize many external sources like shodan, dnsdumpster, webarchive, etc.
sudomy.sh -d example.com
Subfinder is a subdomain discovery tool that discovers valid subdomains for websites by using passive online sources.
subfinder -d example.com
Puredns is a fast domain resolver and subdomain bruteforcing tool that can accurately filter out wildcard subdomains and DNS poisoned entries. It uses massdns, a powerful stub DNS resolver, to perform bulk lookups. With the proper bandwidth and a good list of public resolvers, it can resolve millions of queries in just a few minutes.
Here’s how to brute force a massive list of subdomains using a wordlist named “all.txt”:
puredns bruteforce all.txt example.com
A tool to find subdomains for your target on GitHub.
Assetfinder helps you find domains and subdomains potentially related to a given domain. Written by tomnomnom. It can search through: crt.sh, certspotter, hackertarget, threatcrowd, wayback machine, dns.bufferover.run, facebook (needs API key and secret), virustotal, findsubdomains.
assetfinder --subs-only example.com
The whole idea of DNS brute-forcing is of no use if you don’t use a great wordlist. Selection of the wordlist is the most important aspect of brute forcing.
It is best if you use a wordlist customized to your target. You may use tools like cewl to generate custom wordlists based on words found on the website of your target.
To create more sophisticated wordlists and uncover “hidden” subdomains, you may use Gotator which is a tool to generate DNS wordlists through permutations.
Some ready-made, large but great wordlists are:
- Assetnote best-dns-wordlist.txt (9 Million)Assetnote wordlists are the best. No doubt this is the best subdomain brute forcing wordlist.
- Jhaddix all.txt (2 Million) Created by the great Jhaddix. Was last updated 2 years ago but still works well.
- Smaller wordlist (102k )Created by six2dez is suitable to be run on home systems.
Dimitris is an Information Technology and Cybersecurity professional with more than 20 years of experience in designing, building and maintaining efficient and secure IT infrastructures.
Among others, he is a certified: CISSP, CISA, CISM, ITIL, COBIT and PRINCE2, but his wide set of knowledge and technical management capabilities go beyond these certifications. He likes acquiring new skills on penetration testing, cloud technologies, virtualization, network security, IoT and many more.