Proxy Settings [ Watch Video ]

Proxy tab allows for configuration of which search engines get queried using proxy servers and allows for scanning of proxy servers to ensure they are alive and non-transparent.

D.R.T. was designed to be a bulk scanning tool, however, in order to ensure that it performs at peak levels certain requirements have to be met.

First off, it's important to understand that most search engines (Yahoo, Google, Alexa, Overture) ban an IP after a large amount of queries. While "large amount of queries" varies from one search engine to another, sending more than 300 queries per day is considered "bulk" and grounds for temporary IP banning.

Temporary IP banning can last anywhere from 5 minutes to 3 hours. Secondary offenses may result in longer bans, and finally permanent bans.

To get around this problem D.R.T. includes proxy server support.

A proxy server is an Internet Server which serves as a bridge between your PC and a site you are querying. The purpose of the proxy server is to hide your IP address from the remote web site so that the queries looks like they are coming from the proxy server. Of course, proxy servers can get banned just as easily as a regular user can and that is why we recommend having at least 20-50 fast, non-transparent proxy servers loaded in the proxy server list.

D.R.T allows for complete customization over which search engines get queried via proxy and even automatically removes proxy servers which perform poorly and slow down the overall search.

Most options in the proxy search engine are self explanatory since they relate directly to which search engine should be queried via proxy, however the "Automatically disable proxies..." feature will be discussed.

The Automatically disable proxies with high failure rate feature, when enabled, will automatically disable proxy servers with failure rates higher than 85%. This ensures that poorly performing proxies get removed from the proxy list (during scan process only, they will still exist in the proxy list) and do not get used during the scan. The auto-disable feature requires at least 10 queries before it considers proxies for removal.

The Maximum Proxy Verify Threads setting determines how many proxies should be checked at a time. The Maximum Proxy Verify Timeout setting determines how long the connection should be kept open (if no response) before D.R.T. determines that the proxy server is not responding or is too slow for use.

Proxy server scanning is an extremely important feature in D.R.T. and it should be understood well.

It is important to understand that loading multiple proxy servers into the proxy list and placing a check mark next to them will result in those check marked proxy proxy servers being used in the scan (if "Use Proxy Servers is checked" is checked). When D.R.T. uses multiple proxy servers it randomizes a different proxy with each query, ensuring that no one proxy gets overused or banned.

Many proxy server list sites exist, however, most of the proxy servers provided by these sites are usually dead, or inactive. This obviously introduces a problem because loading dead proxy servers into D.R.T. server list and attempting to use these servers will result in mostly "retries" and error results because no queries will have gotten through to the actual search engines using non-function proxies.

This is why D.R.T. scans proxy servers to ensure that they work. Another very important feature is scanning proxies for transparency. Many proxy servers are transparent, meaning that they tell the remote server your real IP address and that you are performing the query via proxy. For D.R.T. purposes, these proxy servers are useless, and they must be identified and removed from the proxy list. Only Anonymous proxy servers should be used.

Domain Research Tool accepts proxy server lists in many different formats, as long as there is only one proxy server per line. Proxies can be pasted directly into the Proxy List by right clicking on the proxy list and selecting "Paste". Sample formats supported:

• ip:port
• ip:port misc information
• ip port misc information

The IP address and Port number are the only two required fields. They can be separated using tabs, spaces, colon. Miscellaneous data before or after the port/ip is ignored.

Click below to watch a sample video of how to find and load proxy servers into D.R.T.

Proxy Usage Flash Video

The above video is a 30 second how-to using Google and some free proxy lists web site. Quality proxy lists produce much more efficient results, are faster, and usually cost a few dollars a month. If you are using D.R.T. for bulk queries it may be worth it to subscribe to a proxy site that provides daily updated proxy servers.

Once proxies are loaded, scanned, cleaned and saved they can be used in the domain scan. Note that D.R.T. will only use proxies that have a check mark next to them. All other proxy servers which are not check marked are skipped during the scan. Remember to save changes to the proxy file after checking/unchecking proxy servers.

D.R.T. Will load the last proxy list each time it is loaded, so there is no need to manually go through this process every time, however, it is strongly recommended that you check proxies often to ensure that they are active (at least once per day).