The Anatomy of a Great data integration
https://telegra.ph/10-Tell-Tale-Signs-You-Need-to-Get-a-New-synthetic-dataset-04-25
This enables Goggle (person-agent identify GoogleBot) to go to just about every web site and Listing, whilst at the same time banning Talk to Jeeves with the web-site wholly