Webbots, Spiders, and Screen Scrapers~tqw~ darksiderg

*******************************************************************************
Webbots, Spiders, and Screen Scrapers
*******************************************************************************
-------------------------------------------------------------------------------
General Information
-------------------------------------------------------------------------------
Type.................: Ebook
Part Size............: 2,424,371 bytes
-------------------------------------------------------------------------------
Post Information
-------------------------------------------------------------------------------
Posted by............: ~tqw~
-------------------------------------------------------------------------------
Release Notes
-------------------------------------------------------------------------------
The Internet is bigger and better than what a mere browser allows. Webbots,
Spiders, and Screen Scrapers is for programmers and businesspeople who want to
take full advantage of the vast resources available on the Web. There's no
reason to let browsers limit your online experience-especially when you can
easily automate online tasks to suit your individual needs.
Learn how to write webbots and spiders that do all this and more:
* Programmatically download entire websites
* Effectively parse data from web pages
* Manage cookies
* Decode encrypted files
* Automate form submissions
* Send and receive email
* Send SMS alerts to your cell phone
* Unlock password-protected websites
* Automatically bid in online auctions
* Exchange data with FTP and NNTP servers
Sample projects using standard code libraries reinforce these new skills. You'll
learn how to create your own webbots and spiders that track online prices,
aggregate different data sources into a single web page, and archive the online
data you just can't live without. You'll learn inside information from an
experienced webbot developer on how and when to write stealthy webbots that
mimic human behavior, tips for developing fault-tolerant designs, and various
methods for launching and scheduling webbots. You'll also get advice on how to
write webbots and spiders that respect website owner property rights, plus
techniques for shielding websites from unwanted robots.
As a bonus, visit the author's website to test your webbots on sample target
pages, and to download the scripts and code libraries used in the book.
Sometasks are just too tedious-or too important!- to leave to humans. Once
you've automated your online life, you'll never let a browser limit the way you
use the Internet again.
Table of Contents
Dedication
ACKNOWLEDGMENTS
Introduction
FUNDAMENTAL CONCEPTS AND TECHNIQUES
WHAT'S IN IT FOR YOU?
IDEAS FOR WEBBOT PROJECTS
DOWNLOADING WEB PAGES
PARSING TECHNIQUES
AUTOMATING FORM SUBMISSION
MANAGING LARGE AMOUNTS OF DATA
PROJECTS
PRICE-MONITORING WEBBOTS
IMAGE-CAPTURING WEBBOTS
LINK-VERIFICATION WEBBOTS
ANONYMOUS BROWSING WEBBOTS
SEARCH-RANKING WEBBOTS
AGGREGATION WEBBOTS
FTP WEBBOTS
NNTP NEWS WEBBOTS
WEBBOTS THAT READ EMAIL
WEBBOTS THAT SEND EMAIL
CONVERTING A WEBSITE INTO A FUNCTION
ADVANCED TECHNICAL CONSIDERATIONS
SPIDERS
PROCUREMENT WEBBOTS AND SNIPERS
WEBBOTS AND CRYPTOGRAPHY
AUTHENTICATION
ADVANCED COOKIE MANAGEMENT
SCHEDULING WEBBOTS AND SPIDERS
LARGER CONSIDERATIONS
DESIGNING STEALTHY WEBBOTS AND SPIDERS
WRITING FAULT-TOLERANT WEBBOTS
DESIGNING WEBBOT-FRIENDLY WEBSITES
KILLING SPIDERS
KEEPING WEBBOTS OUT OF TROUBLE
Appendix. PHP/CURL REFERENCE
Appendix. STATUS CODES
Appendix. SMS EMAIL ADDRESSES
Index
Product Details
* ISBN: 1593271204
* ISBN-13: 9781593271206
* Format: Paperback, 306pp
* Publisher: No Starch Press San Francisco, CA
* Pub. Date: April 2007
-------------------------------------------------------------------------------
Install Notes
-------------------------------------------------------------------------------
CHM Reader