Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It's called "scraping" and I've done it lots of times with python, very easy. Don't bother with those other specialized, non-python, frameworks that people are suggesting.

http://wwwsearch.sourceforge.net/mechanize/

And if you need to do complicated html parsing in combination with that:

http://www.crummy.com/software/BeautifulSoup/

From there, it's cake.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: