Use google as a tool for development
When i am building websites, i sometimes need to run big scripts, sometimes these scripts take several days up to weeks to finnish running. The problem with running big scripts are that either you have to build in a delay in the script that is running, that will make your script take even more time. Your other option will be that you run your script with full access to resources, and that all your sites on the dedicated server will be slower, due to the extensive resourece usage of this script. I have a new aproach for this, why not let google build the pages when google index them ? This way only the indexed pages are constructed, google will crawl most of the pages, the first time someone else crawl that page, it will be built (fair enough, it will take a bit more time then if it would be already built, but you will not have the same problems as if a script that have run for a week fails or times out.