blogging and scraping » watir

The poor support of the firewatir 1.6.5 to Chinese characters

leleba — Thu, 07 Jan 2010 09:05:41 +0000

I cannot figure out how to enter the Chinese characters into the textfield of a webpage by firewatir 1.6.5, I tried to change the encoding of the .rb file and the encoding of the firefox browser, but failed.

require ‘rubygems’
require ‘firewatir’

ff = FireWatir::Firefox.new
ff.goto “http://www.google.com“

sleep 1
ff.text_field(:name, ‘q’).set(‘?’)

For Watir 1.6.5, the Chinese charaters can be supported by replacing the script WIN32OLE.codepage = WIN32OLE::CP_UTF8 with WIN32OLE.codepage = WIN32OLE::CP_ACP in the file of C:\Ruby\lib\ruby\gems\1.8\gems\watir-1.6.5\lib\watir\win32ole.rb

Check the version of the watir and the firewatir

leleba — Thu, 07 Jan 2010 08:35:14 +0000

We can check the version of the watir and firewatir by executing an rb file with the following script:

require ‘watir’

require ‘firewatir’

puts Watir::IE::VERSION

puts FireWatir::Firefox::VERSION

It is safe require “rubygems” in the begining of the watir codes

leleba — Mon, 28 Dec 2009 04:39:10 +0000

Maybe it is safe to add the code of require ‘rubygems’ in the beginning of the watir codes, otherwise, strange results may happen sometimes .

the fucntion ie.collect in watir

Tue, 10 Nov 2009 00:44:51 +0000

Just copy a coding found in the internet for future reference,

$ie.divs.collect {|div|
if div.class_name == “data”
puts div.text
end
}

The fast way to fill a textfield in watir

Mon, 09 Nov 2009 15:30:55 +0000

It is quite slow to fill a text field by ie.text_field().set(content), the fast way is by ie.text_field().value = content.

There is no property of class for iframe in watir

Mon, 02 Nov 2009 15:21:24 +0000

I encountered an iframe in a webpage which say “iframe class = “the_iframe”, I tried but failed to access the iframe by:
ie.frame(:class, “the_iframe”)

and then I figured out there is no property of class for iframe in watir, and find the iframe by index:
ie.frame(:index, 1)

we can use ie.show_frames to find how many frames are there in a webpage.

Be careful to “include Watir”

Thu, 29 Oct 2009 06:12:39 +0000

When we have the “include Watir”, then we cannot load another ruby file.

I spent more than 1 hour to figure out this.

watir to count the links in a search list on a webpage

Tue, 27 Oct 2009 11:10:25 +0000

Some times I need to check the amount of the links in a search list, at first I try and failed by:
ie.links(:id, /bra bra/).length

and then I find a solution by:
ie.div(:id, “searchList”).links

and then by googles, I find a better solution:
links_searchlist = ie101.links.find_all { |link| link.class_name == ‘permalink’ }
puts “the links in the searchlist:”
puts links_searchlist.length

and in another case, following code also works:
searchlistlinks = ie102.links.find_all { |link| link.id =~ /hitURL/ }
links_searchlist = searchlistlinks.length
puts links_searchlist

following can also work:
searchlistlinks = ie33.links.find_all { |link| link.href =~ /something/ }
but be careful that the following (url) will not work:
searchlistlinks = ie33.links.find_all { |link| link.url =~ /something/ }

Recently I found the best solution by google:
links = ie51.div(:id, ‘threadlist’).html.scan(/something/).count
puts links

Gems for web scrape

Fri, 23 Oct 2009 06:28:40 +0000

I just put the names of the gems here for future reference:
hpricot
nokogiri
mechanize
webrat
scrubyt

The most useful site for watir

Fri, 23 Oct 2009 03:37:43 +0000

I always googles when I encounter any problem related to watir, but after these years’ practice, I will state that the most useful site for watir is http://wtr.rubyforge.org/rdoc/, most answer is already there.