Grubby, but lustrous

I was very proud of my Pages to rtf script, and then I upgraded to Leopard last night and it stopped working! In my search for the answer I came across this post, which solves the problem more thoroughly, though you need to combine it with information from here as well.

In case you don’t want to do this yourself, here’s a script that will start Pages, prompt you for the type of export you want, and then prompt you for a destination directory, and it works under Leopard. I told you it was more thorough than my last solution

on open theFiles
	tell application "System Events"
		if process "Pages" exists then
			display dialog "whoops, please close Pages before running droplet!"
		end if
	end tell
	tell application "Pages"
		activate
		delay 1
		close front document
		set theList to {"doc", "rtf", "pdf", "txt"}
		set theType to (choose from list theList OK button name "Select" with title "Pages Export" with prompt "Choose one or more formats to export using Pages" with multiple selections allowed) as text item
		set s to theType as string
		set theLocation to choose folder
		set theLocation to theLocation as string

		repeat with aFile in theFiles
			open aFile

			if theType contains "doc" then
				set asType to "SLDocumentTypeMSWord"
				set docName to name of front document

				-- Remove .pages extension.
				set prevTIDs to AppleScript's text item delimiters
				set AppleScript's text item delimiters to ".pages"

				-- Add .doc extension.
				set docNameNew to first text item of docName & asType
				set AppleScript's text item delimiters to prevTIDs

				-- Save file to Desktop.
				set docPathAndName to theLocation & docNameNew
				save front document as asType in docPathAndName

			end if
			if theType contains "rtf" then
				set asType to "SLDocumentTypeRichText"

				set docName to name of front document

				-- Remove .pages extension.
				set prevTIDs to AppleScript's text item delimiters
				set AppleScript's text item delimiters to ".pages"

				-- Add .doc extension.
				set docNameNew to first text item of docName & asType
				set AppleScript's text item delimiters to prevTIDs

				-- Save file to Desktop.
				set docPathAndName to theLocation & docNameNew
				save front document as asType in docPathAndName
			end if
			if theType contains "pdf" then
				set asType to "SLDocumentTypePDF"
				set docName to name of front document

				-- Remove .pages extension.
				set prevTIDs to AppleScript's text item delimiters
				set AppleScript's text item delimiters to ".pages"

				-- Add .doc extension.
				set docNameNew to first text item of docName & asType
				set AppleScript's text item delimiters to prevTIDs

				-- Save file to Desktop.
				set docPathAndName to theLocation & docNameNew
				set s to save front document as asType in docPathAndName

			end if
			if theType contains "txt" then
				set asType to "SLDocumentTypePlainText"

				set docName to name of front document

				-- Remove .pages extension.
				set prevTIDs to AppleScript's text item delimiters
				set AppleScript's text item delimiters to ".pages"

				-- Add .doc extension.
				set docNameNew to first text item of docName & asType
				set AppleScript's text item delimiters to prevTIDs

				-- Save file to User Specified Location
				set docPathAndName to theLocation & docNameNew
				save front document as asType in docPathAndName
			end if
			try
				close front document saving no
			end try
		end repeat
		quit
	end tell
end open

“In 1957 some of us put forward a partially complete theory of the weak force, in disagreement with the results of seven experiments. It was beautiful and we dared to publish it, believing that all those experiments must be wrong.

In fact, they were all wrong.”

I love that in maths and physics truth and beauty are so frequently aligned, and that misalignment often indicates immature understanding. A new insight can reveal new beauty.

I think the same is true of programming. Your code should be at least as elegant as your problem domain (I can’t do anything about accounting *s*).

My current assignment makes me a Mac weenie in a Microsoft world, so while I’m happily tootling along in Pages I need to be able to share stuff with my colleagues. At the moment I’m the primary author of 30+ short course descriptions, stored in separate Pages files, and I’ve been exporting them to RTF when I need to share them. That was initially ok, but got out of hand as the number of files grew, so yesterday I was introduced to AppleScript.

I know nothing about AppleScript, but I was able to find a script on the web (thanks again, Google) that did most of what I want. I’ll show you the script, then tell you the parts that weren’t obvious to me and slowed me down

on open theFiles
	tell application "Pages"
		repeat with aFile in theFiles
			open aFile
			set docName to name of front document
			-- Remove .pages extension.
			set prevTIDs to AppleScript's text item delimiters
			set AppleScript's text item delimiters to ".pages"
			-- Add .pdf extension.
			set docName to first text item of docName & ".rtf"
			set AppleScript's text item delimiters to prevTIDs
			-- Save file to Desktop.
			set docPathAndName to (path to desktop as string) & "rtfs:" & docName
			save front document as ".rtf" in docPathAndName
			close front document
		end repeat
	end tell
end open

My first challenge was that I couldn’t figure out how to get this to run! Pressing “Run” in ScriptEditor did nothing. I ended up saving it as an applet (use “Save As”, selecting FileFormat => application), and then I could drag and drop the files I wanted to convert onto the applet. Once I put the applet in the dock, that was pretty convenient.

The second thing that slowed me down was figuring out how to save the results to a folder (the original script saved to the desktop). What I needed was the bit ‘ & “rtfs:”‘ while setting the docPathAndName. Here I’m dealing with a string, and it already has a trailing “:” - I kept trying to add another one, and AppleScript gave me a message about the target not being writable, rather than not existing, which lead me astray for a while.

Anyway, now I can select my Pages files, drag and drop them onto the applet in my dock, and the rtf files appear in a folder on my desktop. Sweet!

When I’m doing agile training, I commonly advise customers to give names to releases and iterations to help them decide which stories belong in which iteration, but it’s not something that I see many customers do and since I rarely take that role not something I do much myself either. But today I got some direct experience!

Cogent is building a software application to help you organize your personal work, and I’m acting in the sponsor role. The details of the functionality are being sorted out by someone else, but I’m the person focussed on the operational requirements and how we get it to the billable stage as quickly as possible (everyone else in interested in this too, but it’s my particular role). Yesterday afternoon I added operational stories to our list, and then prioritised the list story by story, and got what looked like a reasonable response.

Today I went back and created some themed releases, and gave them names.

1. Homely - Ugly, but interesting. This release is usable, but won’t win any design awards. We want it first because we’re using the application internally now, so functionality is important. This is the release we’ll pass to “friendly” testers for feedback, so it needs to support a small set of concurrent users without getting too slow. Codename : Homer
2. Prettified - Like Homer, but certainly nicer looking and generally more usable. We still won’t be confident in our backup plan, nor have stress tested the application, but it will look professional and be generally usable. Codename : Marge
3. Robust - we need this one to be tough, so it can withstand the abuse of a lot of people. This is the point where we can do an open beta - still not charging, but giving the general public access. Codename : Bart
4. Billable - something that we feel we could reasonably charge money for. Codename : Lisa

With these releases in mind I went back and looked at my priorities again, and found that some of them were quite different. If you are a customer on an agile project, I strongly encourage you to do the same thing - you may be surprised by the difference it makes.

“Simplicity and elegance are unpopular because they require hard work and discipline to achieve and education to be appreciated.” - Edger W Dijkstra

We’ve been asked to produce some “nice” reports in PDF for a client, and getting it all done has been quite an adventure. We use Ruport for the reporting, which unfortunately means we also need to learn quite a bit about PDFWriter to get everything looking spiffy, and the report rendering does take a while, but we had it all working in a controller yesterday and things were looking good. Then we deployed the application to EngineYard.

The first thing that happened was that we got timeout errors that looked like this (courtesy of ExceptionNotifier):

A Mongrel::TimeoutError occurred in reports#cost_detail_pdf:
Mongrel timed out this thread: shutdown
/usr/lib64/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:221:in `within'

The support folks at EngineYard suggested that we shouldn’t be running PDF generation from within our controller, and that we should use something like background job, but since we wanted to report status as the job proceeded and we only had a small number of jobs, we went with BackgroundRb instead. (If you care about the difference, look for ‘Backgroundrb and Bj serve different purposes’ in this thread).

BackgroundRb works great once you get it going, but there were examples in the documentation that simply didn’t work for me. Here’s what worked for me in the end:

In my controller:

  MiddleMan.new_worker(:worker => :cost_detail_pdf_worker, :job_key => current_user.id)
  MiddleMan.ask_work(:worker => :cost_detail_pdf_worker, :job_key => current_user.id,
                                          :worker_method => :build_report,
                                          :data => [session[:production_id], file_name])

and my worker:

class CostDetailPdfWorker  calculated_report, :template => :default)
    register_status("Finished Rendering")
    File.open(file_path, "w") do |file|
      file << pdf
    end
    register_status("File available")
  end

end

The next part of the adventure was getting this running on our Slicehost slice. Unfortunately, BackgroundRb requires Ruby 1.8.5, and we only had 1.8.4 (on Ubuntu 6.06 LTS). Apt-get for Ubuntu 6.06 doesn’t include Ruby 1.8.5, so I needed to install Ruby 1.8.5 from source using these excellent instructions.

Unfortunately, I also needed to reinstall all our gems to go with the new version of Ruby. This was mostly stragithforward, except for Postgres. The postgres gem wouldn’t build the native extensions, failing to find pg_config. These instructions were close, but before that would work I also needed to do “apt-get install libpq-dev” (you might also need libpgsql-ruby and/or libpgsql-ruby11.8, but I already had those installed).

Ok, looks pretty good now. I started the backgroundrb server manually as a test, and could produce my pdf files, so the next step was to move to our production environment at Engineyard. This was the first time I tried to stop and start the BackgroundRb using Capistrano - unfortunately you can’t simply repeat the command lines in your Capistrano tasks. Fortunately, there’s also a description of some tasks that do work (make sure you use the start task in the comments at the end of post).

All in all, a nice outcome but with much more pain than I would have liked.

There’s been a recent discussion in the Melbourne XP Enthusiast Group (MXPEG) mail list about whether or not TDD ‘works’, with part of the thread also being about Story Driven Development (SDD). Some of the folks have reported a lot of success using SDD and its associated tests (that I would call acceptance and integration tests), to the point where they don’t use unit tests at all. The impression I get is that with this approach they’re working predominantly top down. I’m glad this works for some people, but I don’t think it would work for me, and I wanted to take a few minutes to say why.

I think it’s important to have both unit tests (developer tests) and acceptance tests (customer tests), whether they’re implemented in the same tools by the same people or quite different tools by quite different people. Notice though that I said “implemented” - to me the two groups of tests have different owners, different perspectives, and serve different purposes.

Customer tests - they tell us if the application is sick or not, but they don’t provide any indication of where or why.

Developer tests - they tell us exactly where something is going wrong, but don’t necessarily tell us what the impact on the overall application would be.

I would like to have both of these, not one or the other. I also find that I personally don’t design entirely either top down or bottom up. I do some top down stuff to get an idea of the problem, then I use that to hypothesise about a design that will meet the requirements. Once I have some confidence that my design looks reasonable, I implement parts of it bottom up, and I want unit tests to confirm that my implementation of each component works the way I expected. I keep switching back and forth between top down and bottom up as I go.

The thing I do badly is that I don’t write acceptance tests until the end (if at all). I know this isn’t what I’m supposed to do, I know it probably detracts from my overall quality, but it is what I find myself doing. So far I haven’t found a tool that makes writing those tests easy enough for me (personal opinion only).

… but Bob Martin did :

One of the developers asked the question point blank: “What do you do when your managers tell you to make a mess?” I responded: “You don’t take it. Behave like a doctor who’s hospital administrator has just told him that hand-washing is too expensive, and he should stop doing it.”

The system I’m currently developing needs to import a variety of data from another system. The data arrives in batches, and within a batch some items might be valid and some might be invalid - I expect that this is a common problem. An item in the batch might also update many different objects within my domain model, and I won’t necessarily know if the item is invalid until I try to commit the changes to the domain model. It became fairly clear that I had some recurring code patterns in my application, so of course I wanted to extract them into some abstraction and Ruby’s reflection gave me a nice way to capture this.

What I needed was to be able to associate a piece of code that created or modified some domain objects with the code that cleaned up the changes to the domain model on rollback (assuming that the database transaction handled the persistence parts). I could have used blocks, but I find that unlabelled blocks aren’t very expressive so I decided to use methods (and their names) instead of blocks. Here’s an example of using my method, #within_transaction_with_rollback; in the example :create_reference_objects and :clear_reference_objects are methods defined somewhere else in the class.

  def process
    within_transaction_with_rollback(:create_reference_objects, :clear_reference_objects)
  end

And here’s the implementation of my method, for those who are interested. I’m sure that there are plenty of other possible implementations

module Transactions

    def within_transaction_with_rollback(transaction_method, rollback_method)
      begin
        within_transaction(transaction_method)
      rescue
        self.send(rollback_method)
      end
    end

    def within_transaction(create_method)
      self.class.transaction do
        self.send(create_method)
      end
    end

end