Fetch URLs as search engine robot ("fetch as")
Show content or headers of a given URL with the HTTP "User-Agent" header set to a search engine robot. Defaults to Googlebot.
Online help:
usage: fetchas [-h] [-r] [-I] [-a ROBOT_AGENT] url
Fetch URLs as search engine robot.
positional arguments:
url
optional arguments:
-h, --help show this help message and exit
-r, --follow-redirects
Follow redirects, if any occur
-I, --headers Show only response headers
-a ROBOT_AGENT, --robot-agent ROBOT_AGENT
ROBOT_AGENT should be one of the following values
(default: 'googlebot'): default, desktop, googlebot,
mobile, googlebot-mobile, googlebot2, bingbot,
bingbot-mobile, browser, iphone, pixel