The vendor review you're failing without knowing it — and the layer that decides whether a crawler gets in.
Reads the response headers and the HTML of one public URL. The results are not stored and no report is published. Sixty tests per hour — the second instrument and each site you add to a comparison each count as one.