DNS and Proxy Bypass — Discover Original IP address of a website.
Most of the time when we are performing a penetration test, we do not find the actual IP address of the website because it is hidden with cloud proxy services.
Sometimes, if I find the original domain name of a website, it may help me discover other websites that are linked to from there and other useful information from original web server which is protected by Proxy server. In this blog post, we will cover methods that you can use to discover the real IP address of a website. We will also cover how to find out if it is hosted on cloud services such as AWS, Azure and GCP.
What I am covering here
Why proxies behind the Website?
Who are the Service providers of Cloud Proxy?
How to find website hosted in public cloud like AWS, Azure & GCP?
Download IP-Ranges from different cloud provider.
Scan the IP-ranges with masscan and validate with httpx
Removes false positive and speed up the scanning process.
Tools to confirm scanned IP address using the Proxy
Why companies use proxies for their website?
By default, the IP of xyz.com resolve proxy servers IP i.e 10.0.1.1 in the example not the real web server IP
How to find real IP address of website during reconnaissance in Pen testing?
For example, xyz.com website is running at ec2 instance of AWS cloud and proxy is configured with Cloudflare.
In that case, The web is running in an EC2 instance with the IP39.6.13.6, but if I check the “A” DNS record, I get an IP inside Cloudflare’s network
How to find xyz.com original IP 39.6.13.6?
https://github.com/carlospolop-forks/hakoriginfinder
hakoriginfinder:
Tool for discovering the origin host behind a reverse proxy. Useful for bypassing WAFs and other reverse proxies.
But challenge with Hakoriginfinder is it makes HTTP requests to hosts and compares the responses with its original — to see if they are potentially matching.
It requires the original IP
Download AWS IP ranges before input into Hakoriginfinder
For download AWS IP ranges, https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html
These IP ranges will have millions of Ips, we need to scan each IP and get open ports. Particularly the port 443 as per xyz.com
To do the mass scan, we requires masscan https://github.com/robertdavidgraham/masscan
This is an Internet-scale port scanner. It can scan the entire Internet in under 5 minutes, transmitting 10 million packets per second, from a single machine.
Usage is like nmap. To scan a network segment for some ports:
# masscan -p80,8000–8100 10.0.0.0/8 2603:3001:2d00:da00::/112
This will:
scan the 10.x.x.x subnet, and 2603:3001:2d00:da00::x subnets
scans port 80 and the range 8000 to 8100, or 102 ports total, on both subnets
print output to <stdout> that can be redirected to a file
To see the complete list of options, use the — echo feature. This dumps the current configuration and exits. This output can be used as input back into the program:
# masscan -p80,8000–8100 10.0.0.0/8 2603:3001:2d00:da00::/112 — echo > xxx.conf
# masscan -c xxx.conf — rate 1000
So, we need to provide the input IP ranges downloaded from AWS https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html
Masscan, we can configure thread size accordingly based on the system resources we are scanning.
Our next step is to confirm web services are running based on scan done by masscan — this reduces false positive
For this we need to use httpx https://github.com/projectdiscovery/httpx
This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.
The filtered output of httpx can be provided to hakoriginfinder to confirm the IP address 39.6.13.6 using cloudflare as proxy.
cat ouput.txt | hakoriginfinder -t 200 -h xyz.comoutput.txt — output filtered from httpx
-t option of hakoriginfinder is # of threads
-h option indicates host
This will find the original IP address 39.6.13.6 running at AWS ec2 instance.
What if we are running our web site in Azure or GCP, and how can we download IP-ranges?
Azure IP-Ranges can be downloaded from here https://www.microsoft.com/en-us/download/details.aspx?id=56519
GCP IP-Ranges can be downloaded from here https://gist.github.com/n0531m/f3714f6ad6ef738a3b0a
Challenges running masscan and httpx
There are challenges for running masscan for millions of IP addresses from the IP-ranges downloaded from cloud provider like false positive.
We can apply filters and make custom scripts to reduce false positive and scan faster.
Removes URLs that redirect or load via SSRF, compare URLs that are similar to the other one by looking at such data as HTML title and response length and URLs to the specified URL by calculating its simhash
Conclusion:
This technique will fail to bypass proxies if an organization has configured its proxy servers with high protection and strict configuration. Most organizations, however, have failed to take this precautionary measure.
The steps above should be fine-tuned in real life because of the time it takes to scan a website and remove false positives.
