K, kids who wants to get geeky?
First for the non geeks lets talk about something you aren’t supposed to know about. Doesn’t matter if you do or don’t but it will provide some interesting results.
Search engines use what’s called robots to find it’s results. Robots will go off into the web like Louis and Clark, and report back it’s findings the search engine would then catagorize them.
The robots.txt file tell those buggers where they are allowed and not, potentially it can inform you where the secret shit is on a website. Most websites have them. This is where I come in, I was searching for info on robots.txt files, and guess what I found on google…
Yeah the whitehouse, I’m not sure if it’s illegal or not to look at it. But the weird thing is it’s showing up in google. It’s not supposed to do that last I checked. Maybe too many people linked to it and it just popped up in it’s results.
Odd shit, play with this one at your own risk, I know nothing.
