Introduction to Harvester

Gathering information about e-mails of employees of an organization can give us a very broad attack vector against the target. This method can be classified under passive reconnaissance since we are not engaging with the target in any way, but would be using search engines to gather a list of e-mails. These e-mail lists and usernames could be used later for social engineering attacks and other brute force attacks. We will discuss this once we get to the exploitation phase. It’s quite a tedious job to gather e-mails one by one with Google. Luckily, we have lots of built-in tools in BackTrack that can take care of this. One of those tools is TheHarvester, written in Python. The way is works is that it the data available publicly to gather e-mails of the target. This tool is available in BackTrack by default under the /pentest/enumeration/google/harvester directory. To run the tool from the directory, type the following command:
./theHarvester.py

Now, let’s say that we are performing a pentest on Microsoft.com and that we would like to
gather e-mail lists. We will issue the following command:

The -l parameter allows us to limit the number of search results; for example, here we have
limited it to 500 by assigning –l 500 command. Along with it, you can see a -b parameter;
this tells TheHarvester to extract the results from Google. However, you can change it to Bing or LinkedIn, and the tool will return the relevant results from the Bing search engine and LinkedIn. You can also use -all parameter to make the tool search for results in all of these websites.

Next, we can search individual e-mails in pipl.com, which is one of the largest, high-quality people search engines, and try to find relevant information.

Through this search, we’ve some interesting information for tharris@microsoft.com. So from just a simple e-mail address, we were able to gather a complete profile.
This information could be very useful in performing social engineering attacks, stressing the fact that humans are the weakest link. With a little more digging, we’ve managed to find the LinkedIn and Facebook account of Tim Harris.

GATHERING WORDLIST FROM A TARGET WEBSITE

After we have gathered e-mail lists from search engines, it would be really useful for us to gather a list of words that we would use for brute forcing purposes. CEWL is another excellent tool in BackTrack, which enables you to gather a list of words from the target website, which can be later used for brute-forcing the e-mail addresses we found earlier. It can be found in the/pentest/passwords/cewl directory.
You can issue the following command in the /pentest/passwords/cewl directory to execute it.
ruby cewl.rb –help

SCANNING FOR SUBDOMAINS

Most Webmasters put all their efforts in securing their main domain, often ignoring their subdomains. What if an attacker manages to hack into a subdomain and uses it to compromise the main domain. Depending upon the scope of the pentest, you might also need to test subdomains for vulnerabilities. A very common way of searching for subdomains is by using a simple Google dork.
Even though you won’t be able to find all the subdomains with this method, you can find some important ones.
Site: http://msn.com -inurl:www

This query is telling the search engine to return results without www, which are normally subdomains. However, it will not be able to find subdomains that have the following pattern:
http://www.subdomain.msn.com
Since, we have already asked Google to return results without www.

THEHARVESTER

TheHarvester can also be used for this task, which uses Google to search for subdomains.
[Harvester Manages to extract Subdomains for Mozilla]

FIERCE IN BACKTRACK

Fierce is also an amazing tool for scanning subdomains. Fierce uses a variety of different methods to enumerate subdomains such as brute force and zone transfer. It is also capable of bypassing CloudFlare protection. Fierce comes preinstalled in BackTrack. It is located in the /pentest/enumeration/dns/fierce directory.

To scan a host for subdomains, you need to issue the following command from the fierce
directory.
./fierce.pl -dns <domain>
As you can see , I have used the –threads parameter and set the value at 1000. This will make it run faster. Initially, it tries to perform a zone transfer. If it fails, it would start brute-forcing the servers.
You can also provide fierce a custom wordlist.
Example
/fierce.pl -dns xyz.com -wordlist <wordlist path>

KNOCK.PY

Knock.py is a tool that has capabilities similar to fierce for determining subdomains. It has a built-in internal list as well as the capabilities of scanning with your custom wordlist. It can also perform zone transfers; for that purpose, you just need to issue an additional parameter (-zt).
Examples
Scanning with internal lists:
Python knock.py <url>
Scanning with custom wordlist:
Python knock.py <wordlist>
Zone transfer file discovery:
Python knock.py <url>-zt

Knock.py has various options, which I will leave for you to explore. You can access its documentation at
https://code.google.com/p/knock/wiki/documentation