Ask Your Question
-1

Best way to integrate web browser with local databases and scripts?

asked 2012-04-12 14:45:02 -0500

tim.landscheidt gravatar image

updated 2012-04-13 17:13:59 -0500

I have several applications where I (at the moment: manually) either copy and paste the URL or the page source to a local (PostgreSQL) database or script for further processing (screenscraping, etc.).

What is the best current practice with Fedora 16 for automating these? I use Konqueror, Firefox and Chrome and all can interact with the web sites. I'd like to add buttons to a toolbar or something similar that when pressed execute a program or put some text in the primary selection.

One example would be to visit http://www.imdb.com/title/tt0383126/epcast as a logged-in user, parse the data and pipe it into my PostgreSQL database.

I see that Firefox has a rather advanced add-on system which also looks very complicated :-). Is there an easier solution with Konqueror or Chrome? I have used Konqueror's "mini programs" (don't know if this is the correct term) in the past but I'm looking for something more powerful.

I don't want to use a local webserver and XSS.

edit retag flag offensive close merge delete

Comments

Please elaborate.

lzap gravatar imagelzap ( 2012-04-13 10:37:57 -0500 )edit

I'm curious: Your question says "what is the best way on Fedora to do this". How do you do it on any other OS?

FranciscoD_ gravatar imageFranciscoD_ ( 2012-05-06 15:24:39 -0500 )edit

@FranciscoD_: I don't :-). I just wanted to avoid answers like: "Install Debian and do X." I wouldn't want to stray away from Fedora for this.

tim.landscheidt gravatar imagetim.landscheidt ( 2017-09-18 13:12:18 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2012-05-18 02:22:23 -0500

vadim.rutkovsky gravatar image

Site scraping is generally a bad idea, but you may try to use Selenium/WebDriver to automate actions on the webpage.

Check out bindings for Java, Ruby or Python

edit flag offensive delete link more

Comments

Why do you say is a bad idea? I have several sites backed and shared with some friends, forever lost in the web and not even available in wayback machine. I've also installed lots of education sites in PCs without even LAN, in order to share it with children without ways at this moment to plug into web.

tezcatl gravatar imagetezcatl ( 2013-07-04 22:31:59 -0500 )edit

Question Tools

1 follower

Stats

Asked: 2012-04-12 14:45:02 -0500

Seen: 167 times

Last updated: May 18 '12