Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Strategy ¶
type Strategy struct {
Hosts []string
}
Strategy is a struct that holds information needed to check if an IP address is associated with a known web crawler. This is often used to differentiate between regular user traffic and automated crawlers, such as those used by search engines.
Fields:
- Hosts: A slice of strings where each string is a host that is known to be associated with a web crawler. For instance, "googlebot.com" for Google's crawler.
func InitStrategy ¶
func InitStrategy() *Strategy
InitStrategy is a package-level function that initializes a Strategy struct with predefined host names of known crawlers. This example uses "sogou.com" as a known crawler host.
Returns: - *Strategy: A pointer to a Strategy instance with prepopulated Hosts field.
func (*Strategy) CheckCrawler ¶
CheckCrawler is a method linked to the Strategy struct that attempts to verify if a given IP address belongs to a known web crawler defined in the struct's Hosts field.
Parameters: - ip: The IP address to check against the list of known crawler hosts.
Returns: - bool: Indicates whether the IP is a known crawler (`true`) or not (`false`). - error: Any error encountered during the DNS look-up process.
The method performs a reverse DNS lookup of the IP address to ascertain if any associated hosts match the ones listed in the Strategy's Hosts field using the matchHost method.
Source Files