Computer Science Canada

Python to Automate Internet Tasks?

Author:  Fexter [ Tue Nov 22, 2011 9:05 pm ]
Post subject:  Python to Automate Internet Tasks?

I want to use Python to automate tasks on the internet, such as getting information from a website, multiplying a number by it, then putting this number back into the page. However, I haven't been able to find anything saying how to do this, other than downloading an extra program for it.

Is it possible to do this without downloading a new program, or is it not possible in the current versions of Python? If it is possible to do it, could someone post where I could find out how to do this, or post some of the code used to do this?

Thanks,
Fexter

Author:  Tony [ Tue Nov 22, 2011 9:19 pm ]
Post subject:  RE:Python to Automate Internet Tasks?

sounds like you are looking for httplib http://docs.python.org/library/httplib.html

Author:  Fexter [ Tue Nov 22, 2011 9:20 pm ]
Post subject:  RE:Python to Automate Internet Tasks?

@Tony
Okay, thank you. Very Happy

Author:  Zren [ Wed Nov 23, 2011 9:50 am ]
Post subject:  RE:Python to Automate Internet Tasks?

If you're not scared at installing or using a library, check out Mechanize and BeautifulSoup. The first is essentially httplib with an upgrade, with built in presets emulating a browser. The latter is extremely useful for breaking up HTML into python syntax.

Mechanize: http://wwwsearch.sourceforge.net/mechanize/
BeautifulSoup: Homepage | Examples

Simple Demo of the two:
Python:

from mechanize import Browser
from BeautifulSoup import BeautifulSoup

url = 'http://google.com'
mech = Browser()
page = mech.open(url)
print "Opened: %s" % url
html = page.read()
soup = BeautifulSoup(html)
tags = soup.findAll('input')
for tag in tags:
    try:
        name = tag['name']
    except KeyError:
        name = ''
    try:
        type = tag['type']
    except KeyError:
        type = 'None'
   
    print "Input(Name: %-12s | Type: %-12s)" % (name, type)

Author:  SNIPERDUDE [ Wed Nov 23, 2011 12:30 pm ]
Post subject:  RE:Python to Automate Internet Tasks?

Those two libraries make me love Python all the more. I'll check them out tonight


: