.Google.com's Gary Illyes exposes Googlebot collects web links rather than "complying with" them, opposing official information.Googlebot collects web links instead of observing all of them in real-time.Gary Illyes discussed this info in Google.com's 'Look Off The Document' podcast.Google's information negates the genuine spider habits.