about img
blogger img

Corys posts image

UnderPaidLoveMonkis posts img

buchos posts img

scotts posts image

Testing's Archive

ROI on software testing

UnderpaidLoveMonki @ 9:56 pm Thursday, March 6th, 2008

Black, president and principal consultant of RBCS, emphasized the cost-effectiveness of creating quality software. “The money you spend to build it right the first time is always less than the money it costs to fix it,” he said.

He listed four ways testing saves money: Finding bugs that get fixed, finding bugs that don’t get fixed, running tests that mitigate risks and guiding the project to success through timely, accurate, credible information for project tracking.

This article explains it very well the Return of Investment on testing early in the software development lifecycle.

Automate, Automate, Automate…!

UnderpaidLoveMonki @ 10:06 pm Thursday, February 7th, 2008

In “Improving problem resolution through automation,”

When developers are distracted and bogged down by trying to identify the root cause of a problem, they are no longer focused on core development activities that truly add business value. And when testers spend time manually gathering problem information and documenting problems, they are no longer focused on uncovering application issues prior to release. This drain on resources results in unfortunate and measurable tradeoffs between release dates, software stability, software performance, software usability and software functionality.

The more inefficient the problem resolution process is, the more painful, visible and costly these tradeoffs become. By merely by automating its application problem resolution processes, this moderately sized application development team can reallocate over $3 million to develop new applications or functionality, improve quality or release applications faster. And these are only the hard savings.

Drop everything you do and automate it….NOW!

TDD really works!

UnderpaidLoveMonki @ 9:04 pm Saturday, January 26th, 2008

In “Research Finds TDD Boosts Developer Productivity,”

“We found that test-first students on average wrote more tests and, in turn, students who wrote more tests tended to be more productive. We also observed that the minimum quality increased linearly with the number of programmer tests, independent of the development strategy employed.”

“The external validity of the results could be limited since the subjects were students. Runeson [21] compared freshmen, graduate, and professional developers and concluded that similar improvement trends persisted among the three groups. Replicated experiments by Porter
and Votta [22] and Höst et al. [23] suggest that students may provide an adequate model of the professional population.”

It’s hard convincing non-agile developers, i.e., waterfall, to practice test driven development (TDD) in order to improve quality of software being developed. Here’s proof published in IEEE on the results of TDD performed on various people. The original article is located here.

Unit Testing

UnderpaidLoveMonki @ 10:13 pm Wednesday, January 16th, 2008

Should a software QA team do unit testing? Hrm, interesting question. Well this article from SearchSoftwareQuality will answer that question and some other questions you may find interesting. :)

TDD and Automated Testing

UnderpaidLoveMonki @ 11:34 pm Thursday, October 25th, 2007

Another good read from Test Obsessed.

Here’s some key points from the blog entry:

1. “TDD tests are unit and sometimes code-level integration tests. But TDD itself is a development approach rather than a testing approach.”

2. Distinction between developer tests and acceptance or system tests — developer tests are code-facing, and a result of TDD; and acceptance tests are business-facing.
**Note: One of the mistakes that Agile teams tend to make is attempting to substitute unit tests for acceptance tests, or vice versa.

3. Being Agile means delivering a continuous flow of value. To achieve that continuous flow of value, we have to use development practices that provide the visibility and feedback needed to make frequent, incremental changes.

4. “Mary Poppendieck says that speed requires discipline. I extend that to agility in general: if you want your team to be Agile, it must also be disciplined. Agile doesn’t mean “do whatever feels good.”

5. “Many teams have found that embedding XP-like practices within Scrum is an incredibly powerful combination.”

6. “The bottom line: automated testing at all levels can be difficult, but the alternatives are worse. Without automated testing, both code-facing and business-facing, the manual testing burden becomes too large, the team begins incurring technical debt, and velocity slows.”

By the way, here’s another good read that came from one of the comments, testing.

Google Test Automation Conference 2007 Videos

Scott Rippee @ 9:46 pm Tuesday, August 28th, 2007

Holy Cow! 27 Videos from the Google test automation conference 2007 with so much important information that my head is spinning (and I only watched a small portion =). Automation, testing strategies, mocking, continuous integration, UI automation, complex distributed system, and many different levels and types of testing explained. These really drive the importance of quality automated testing in many different forms that really push software and systems from many different angles.

The keynote gives a good overview of overall stratagies and the importance of processes like quick build / test turnarounds and developer testing.

Pick Your RoR HTML Parsing Poison

Scott Rippee @ 4:07 pm Thursday, July 5th, 2007

Ruby HTML parsing has been keeping me quite entertained frustrated lately, so I thought I'd share some thoughts. There are a couple of instance in your rails app when you'll want to parse HTML

  1. Automated functional/controller testing
  2. Screen scraping

Functional Testing

The standard method of verifying aspects of resulting HTML in your functional test is HTML::Selector. It's simple, powerful, and baked in. Agile Rails 2nd does a great job of explaining how it's used in functional tests.

  1. def test_add_no_name
  2.   post :add, :color => { :name => '', :hex => '#123456' }
  3.   assert_template 'add'
  4.   assert_select "div[id=errorExplanation]" do
  5.     assert_select "ul" do
  6.       assert_select "li", 'Name is not present'
  7.     end
  8.   end
  9. end



Several options are available, but oh so popular is why's Hpricot. It's fast and enjoyable (although I experienced no joy while learning how to use it =) It also happens to be used in some of the other scraping/navigating libraries (WWW::Mechanize [rdoc] and scRUBYt!).


Some Thoughts...

So if your just concerned with testing use HTML::Selector and the built in asserts. If you have to do very basic screen scraping I would also suggest going with HTML::Selector (as long as speed is not an issue and the scraping is basic) with open-uri or curb for fetching the pages.

For more serious screen scraping bust out Hpricot and if you need to navigate pages via automation use WWW::Mechanize (Mechanize also uses Hpricot so all of that Hpricot knowledge you've absorbed is directly applicable. Mechanize is Hpricot with the ability to click). Don't worry about scRUBYt!. It's more of a pain to figure out than it's worth (but maybe I'm wrong about it. Any good examples/write-ups?).

Hpricot with CSS selector

  1. divs = (doc/"div[@style*='font-weight:'][text()*='$'").inner_html
  2. divs.each do |div|
  3.   if div =~ /\$[0-9]?[0-9]\.[0-9][0-9]/
  4.     self.price = div.to_s.sub('$', '')
  5.   end
  6. end

Hpricot search with XPath

  1. require 'hpricot'
  2. require 'open-uri'
  3. doc = Hpricot(URI.parse("http://google.com/").read)
  5. doc.search("/html/body//p")
  6. doc.search("//p")
  7. doc.search("//p/a")
  8. doc.search("//a[@src]")
  9. doc.search("//a[@src='google.com']")

Using Mechanize to do a search on google

  1. require 'rubygems'
  2. require 'mechanize'
  4. agent = WWW::Mechanize.new
  5. agent.user_agent_alias = 'Mac Safari'
  6. page = agent.get("http://www.google.com/")
  7. search_form = page.forms.with.name("f").first
  8. search_form.q = "Hello"
  9. search_results = agent.submit(search_form)
  10. puts search_results.body

Note that Hpricot lets you use a CSS method of selecting and an XPATH method. Use XPATH if you already have experience otherwise the CSS method is more intuitive.

If you go with XPATH grab the XPather firefox plugin and use it with the DOM Inspector. Also, it works with the firebug firefox plugin. I'm still in awe that it worked when I tried. :) To do this, use firebug to "inspect", choose an element, right click on the page and select "Show in XPather". XPather will open with the selected element locked and loaded.

Finally, if your a Hpricot wiz forget about HTML::Selector and put Hpricot to work for view validation in your functional tests. See this great write up, Testing your Rails views with Hpricot, which demonstrates this elegant solution.

  1. assert_equal "My Funky Website", tag('title')
  2. assert_equal 20, tags('div.boxout').size
  3. assert_equal 'visible', element('div#site_container').attributes['class']