Web Development by Alistair Robinson

« Blog home

Goodbye and Good Riddance to Word Processors: A Better Way To Write Proposals


January 24th, 2012 2 Comments

A few days ago I tweeted:

“Writing a proposal. Oh boy do I hate word processors. There must be a better way.”

It turns out there is, though I ended up forging my own path to suit my skills and working methods. A little background…

I need to produce decent looking proposals for web development projects. Normally I bite the bullet and suffer the awkwardness of Word or OpenOffice or LibreOffice or something, and then export to PDF. It’s all very clever: it took some great minds to produce this software. But it really goes against the grain. It feels so wrong.

I like plain old text, and as a web developer I’m used to defining the style of documents by hand, in CSS, rather than using another program to do it for me, as I write. I would go further and claim that this is not just a matter of personal taste: word processors are bad for the world. Sure, people are used to them, and they cannot imagine a better way, and – most importantly – there is probably no equally mature, friendly, well-supported and funded software in place to allow for a move away from them as things stand. But it does not follow from this that they are good.

What’s Wrong With Word Processing?

The name, to start with. But most of all WYSIWYG. One thing I’ve learned from web development is the importance of respecting the logical separation of style and content. WYSIWYG is not so respectful. Everything happens in the same place and at the same time. Effectively, documents are typeset as you go, character by character. This is silly.

Formatting your document’s style has nothing to do with its content. What you are trying to say is a different concern from how it should look. And I do not mean that the two human operations are more conveniently handled by separate software operations. I mean that they are different human, conceptual, natural concerns. Of course it has to look good (although a. it doesn’t always have to, and b. even plain text looks way better than a clipartified Word document). Yes, of course your document has to conform to a company standard, or your own standard, and how it looks might be partly what you intend to convey. But all this precisely means that these standards and styles ought to be defined separately, beforehand, leaving you to get on with the writing: to concentrate on what it is you have to say. It is the difference between talking to somebody at a party and deciding what you are going to wear to that party.

Using WYSIWYG software such as the horribly feature-rich Word is like going to a party with several changes of clothes and proceeding to change them every time you think that what you’re saying doesn’t go with stripes. Sure, do the best you can to look good, but prepare for that, by going shopping, trimming your beard, and putting on your best Y-fronts. After all that’s done you can get on with talking to people and getting drunk, safe in the knowledge that you’re looking great. Underdressed? Well, if you didn’t care enough to prepare, to research, and to tune your attire accordingly, then you shouldn’t care about being underdressed.

Now, moving on from that straining analogy, the biggest benefit of separating style and content is that content becomes independent of how it is presented. If standards of presentation change, or a piece of writing needs to be reproduced in different formats, or you’ve finally got tired of Comic Sans, then you shouldn’t be going anywhere near your content to achieve the desired formatting changes. But word processors force you to do this. You actually have to open and edit your document, just to re-style it. This wrongheadedness has been spectacular in its worldwide detrimental impact.

(Note that I’m quite ignorant of Word and the no-doubt very sophisticated setups that can be achieved in organizations where stanadardized presentation is important. For all I know maybe you don’t always have to go back and edit your document or re-assign a template of whatever. However, achieving this is just a workaround, and likely would not have a perfect success rate; it seems clear that the default expectation of the software is that you do mix up content and presentation.)

And faced with the option, the expectation, to fiddle about with fonts and bullets, it takes discipline, imposed from above or by yourself, to beat down that natural inclination. This is not a matter of freedom. In my case, I have to struggle with formatting issues almost every time I write – or rather, process – one of these documents, because I might be using different word processors on different platforms with different fonts or whatever. This should not even be an issue, because we already know, deep down, that style and content are logically separate categories (note that this does not mean that they do not sometimes overlap). And in a company environment, standard templates for Word and detailed strictures and guidelines from management on formatting, fonts and colours, are awkward and only partly successful workarounds for a problem that would not exist without WYSIWYG.

The writer of a document has expertise in the subject addressed by that document. They may not have expertise in typography and layout, and usually they do not. The latter, then, becomes nothing but a distraction and a huge waste of time. And … I didn’t ask you to send me a bloody Word attachment in the first place!

The natural paradigm for me, which I get from web development, is to write stuff in a text file, and to define how it looks in another text file, and only then to view how it will look to the document’s recipient. I believe this should go for print documents as well as HTML web pages.

It is already the way of things in printing, publishing and professional writing. I don’t believe Proust was worrying about typefaces when he painstakingly described the memory of a hawthorn bloom, and the prevalence of Word has not made this any less true of professional writers today. Why should any writer, or anyone who becomes a writer for brief periods – whether of poems or proposal documents or annual reports – have to worry about such things? Why has the division of labour, both human and machine, which is so thoroughly advanced in the modern world, been bypassed in this case?

Lightweight Markup

A few years ago I discovered Textile, which is a lightweight markup language. What a Godsend. I can get on with writing:

h1. This is The Main Heading

For a long time I used to go to bed early. Sometimes, when I had put out my candle, my eyes would close so quickly that I had not even time to say "I'm going to sleep." And half an hour later the thought that it was time to go to sleep would awaken me; I would try to put away the book which, I imagined, was still in my hands, and to blow out the light; I had been thinking all the time, while I was asleep, of what I had just been reading, but my thoughts had run into a channel of their own, until I myself seemed actually to have become the subject of my book: a church, a quartet, the rivalry between François I and Charles V. This impression would persist for some moments after I was awake; it did not disturb my mind, but it lay like scales upon my eyes and prevented them from registering the fact that the candle was no longer burning. Then it would begin to seem unintelligible, as the thoughts of a former existence must be to a reincarnate spirit; the subject of my book would separate itself from me, leaving me free to choose whether I would form part of it or no; and at the same time my sight would return and I would be astonished to find myself in a state of darkness, pleasant and restful enough for the eyes, and even more, perhaps, for my mind, to which it appeared incomprehensible, without a cause, a matter dark indeed.

h2. This is a Secondary Heading

Furthermore, each one of the following is an item in an unordered list:

* Something
* Something else
* And another thing

As Oscar Wilde "once said":http://upword.com/wilde/dorgrayp.html,

bq. No artist desires to prove anything. Even things that are true can be proved. No artist has ethical sympathies. An ethical sympathy in an artist is an unpardonable mannerism of style. No artist is ever morbid. The artist can express everything.

Now for some Latin. Lorem ipsum dolor sit amet, duo id viris posidonium signiferumque. Maluisset patrioque vis ad, eruditi imperdiet ex pro. At eirmod luptatum expetenda mei, vel impetus meliore oporteat ex, case postea vivendo per eu. Enim prima ridens an mel. Ne has clita scripserit, eam eu veri mundi dissentiet. Eros torquatos sed cu.

All in a text editor. You know, like Notepad only with nice colours and productivity tools. There’s nothing and nobody messing around processing what I write. Processing comes later, based on separate settings that I have set up for the scenario.

My Great Solution Which, Though Great, Could Most Probably Be Vastly Improved Upon

Up till now I’ve only really used Textile to generate HTML for display on web sites, usually blog posts, but now that I’ve (mostly) moved away from Windows and on to Linux, manipulating text becomes all the more easy. I’ve created a nice wee setup to generate PDF documents styled with CSS straight from a Textile file. I should stress that this involves the terminal and bash scripts, so it’s probably for geeks only.

The upshot is that when I want to write a proposal, I just run a couple of commands to set up a proposal project based on a template, and then get down to writing my document in Textile, and when I want to see what it’ll look like I just run ~/gopdf.sh, which generates a PDF.

The gopdf.sh bash script does three things in order:

A. Generates an HTML file from the textile file using pandoc
B. Generates a PDF from the HTML file using xhtml2pdf
C. Opens the PDF

pdf screenshot

(If I had actually got to the stage where I’ve built up a nice PDF stylesheet I would have shown you something more fancy, but there you go. And the sharp-eyed among you will notice that the line of text immediately following the list is squashed up to it. This might be a problem with pandoc’s understanding of textile, which is not perfect, favouring the alternative Markdown as it does. You can include HTML in among the Textile markup, so a line break here is easy to achieve, or else a CSS bottom margin would probably do it, and would probably be the better choice.)

What it Allows Me to Do

Before I describe how I make all this possible, here in more detail is what I now do every time I get a new client who wants a proposal.

One

So I meet with Bob of Bob’s Cake Company fame, and he tells me he wants a cake web app. I say sure, no problem, then I go home, start up Linux and create a folder called bobs_cake_company, and cd into it. Then…

~/startdoc.sh proposal
subl .

The first command copies over my standard proposal template folder to bobs_cake_company as a sub-folder called “proposal”. The second one just opens the current folder, bobs_cake_company, in Sublime Text 2 (I seem to remember doing some things to get that “subl” command working, but I’m not sure what. Probably just a symlink.)

Two

I write the document in the file that is there waiting for me, proposal.textile. Remember, by writing I just mean writing. Not fiddling with margins or choosing bullets or struggling with alignment issues or fixing pasted text or making sure that the screenshot isn’t blurry or anything else like that.

Three

In the terminal…

compass watch

This generates a stylesheet from the SASS files in which I define the CSS stylings. Full disclosure: currently my method is in its early stages so I don’t yet have a good standard CSS stylesheet for PDFs. I’ll be building this as I go along for a project or two before I can just leave it alone. So right now I’m actually running this Compass watcher when I begin writing, because I’m styling as I go, like a human word processor. Not for long.

Four

Then in the terminal (in a new tab):

~/gopdf.sh proposal

This creates a PDF (or overwrites the one that already exists), and then opens it in the default PDF viewer. Thereafter I just run “~/gopdf.sh proposal” whenever I want to see my changes in the PDF.

Five

Send proposal to client and have a cigarette.

Making it All Possible

The process I’ve just described depends on a few pre-existing elements:

  • pandoc
  • xhtmlpdf
  • startdoc.sh bash script
  • gopdf.sh bash script
  • a proposal_template folder with goodies in it
  • Compass (optional)
  • Sublime Text 2 (optional)
  • A dangerous, expensive and anti-social nicotine addiction (optional)

You do the following stuff once only.

Get pandoc: Pandoc is a Haskell library that converts between lots of different kinds of documents. I would have used it alone had it been able to convert Textile and CSS to PDF directly. (I couldn’t make it work and concluded that pandoc couldn’t do it, but I could be wrong.)

Installation instructions are here

Get xhtml2pdf: xhtml2pdf is a python package that generates PDF files from HTML and CSS. It can be installed with easy_install or pip and also available to download here

If you are so inclined, get Compass: Compass is a CSS authoring framework built on Ruby and SASS. You write in SASS and Compass generates CSS files for you. The latter are what you actually run on the server. I’ve been using it for a year or so and I love it. One of the reasons I like it is just because I can avoid the curly braces and semi-colons of CSS itself. I’m addicted to significant whitespace. (Though it’s worth noting that they’ve been going after designers and have opted to make their SCSS syntax the default because SASS looks too much like programming).

gem install compass

More details on installing Compass here

If you need a great text editor, get Sublime Text 2:It’s all the rage, and for good reason. Get it here

Create startdoc.sh:

 #!/bin/bash

cp -r /home/alistair/projects/proposal_template $PWD/$1
cd $1
mv proposal.textile $1.textile

This copies the proposal template folder, renames it depending on your argument, and also renames the .textile file.

Create gopdf.sh:

#!/bin/bash

pandoc --email-obfuscation=none -s -S -c stylesheets/print.css $1.textile -o $1.html -s -S -c stylesheets/print.css $1.textile -o $1.html
python makepdf.py $1.html $1.pdf
gnome-open $1.pdf

This is where the action happens. It runs pandoc to convert from Textile to HTML – along with print.css – and then runs makepdf.py (which in turn runs xhtml2pdf, as we shall see) to take proposal.html and generate proposal.pdf.

Create the proposal_template folder

We need to create a standard proposal template that will thereafter be copied over to a new project any time you run ~/startdoc.sh. It consists of a folder structure which looks like this:

proposal_template
   |-documentation
   |-images
   |-sass
     ---print.sass
   |-stylesheets
   |---fonts
   config.rb
   makepdf.py
   proposal.textile

The folder sass contains the Compass SASS file; stylesheets contains the CSS that is produces from the SASS, namely print.css; config.rb is the Compass config file; and makepdf.py runs xhtml2pdf, but is itself executed by the gopdf.sh bash script. The folder fonts contains font-files that can be included in the PDF by using @font-face in the stylesheet.

Once all this is copied over by startdoc.sh, proposal.textile is where all the writing happens.

The Python script, makepdf.py, looks like this:

from xhtml2pdf import pisa
import sys

pdf = pisa.CreatePDF(
  file(sys.argv[1], "r"),
  file(sys.argv[2], "wb")
  )

pdf.dest.close()

This is called in gopdf.sh, specifically this line:

python makepdf.py $1.html $1.pdf

So running ~/gopdf.sh proposal will get pisa to take proposal.html and generate proposal.pdf. Pisa, by the way, is the original name – or central core – of xhtml2pdf.

In the near future I hope to fill the standard print.css with lots of proposal-specific stylings, and proposal.textile will probably contain my standard section headings and table of contents.

By the way, when creating the standard template, the Compass stuff can be set up automatically by running the following command.

compass create --syntax sass

(If you like the SASS syntax, that is)

WordPress

I’m writing this in Textile right now, but it struck me that I only had a WordPress blog to post it to. I can use Textile for my philosophy blog, because I built the engine myself in Django, but I hadn’t got far in the past trying to get Textile working in WordPress. Thankfully, Veeti Paananen has provided a plugin, Simple-Textile which seems to work nicely.

It’s really time to move this blog, and a redesign is probably overdue.

Afterthoughts

If you’re not a coder, but you’re an author or just do a lot of writing, and you don’t have to worry too much about making your words look good enough for Bob of Bob’s Cake Company to print out — but you do want a nice, pleasing, peaceful, distraction-free writing environment, then there are some nice minimalist editors out there at the moment, such as OmmWriter and WriteRoom.

I get basically the same thing in Sublime Text when I put it in “distraction-free mode”, so my programming and web design editor also functions as a nice place to write about Wittgenstein, Heidegger, Karl Marx and the evils of traffic control.

But when it comes to writing something big or complex like a book or a thesis, folks sometimes opt for specialist software, such as editors that revolve around the LaTex typsetting markup system. These are usually built on the same paradigm that I’ve been describing, namely the separation of style and content, which is especially important with documents primarily intended for professional print publishing. Word processors are strictly for home and office use, and I imagine those in the world of printing never touch them with a bargepole. It must be galling that word processors are these days thought to be the last word in setting and styling type, because while they have cast that noble profession into obscurity – though certainly not into obsolescence – their own ability to do it well is tragically lacking. Next time you’re in a print shop ask them what they think of Microsoft Word, so long as you don’t find anti-Microsoft rants boring.

Thanks to all of the wonderful and generous programmers out there without whom I would likely still be shouting at Word like a madman.

Further Reading

I’m not the only one, you know.

Why Do I Hate Word Processors?
Word Processors: Stupid and Inefficient
What has WYSIWYG done to us?

Tags: , , , , , , , , , ,
Posted in Productivity, python, technology, web development | 2 Comments »

2 Responses

  1. Andy Murdoch says:

    I totally agree this is a better method than traditionally ‘word processing’ a document.
    Previously I’ve used raw HTML and found it more convenient (and with fewer formatting surprises) than Word. Your set up looks even better.

    It is in theory possible to make Word create a structured document where you define styled regions, e.g. headings, although it is possibly more trouble than it’s worth because you have to be constantly vigilant that you’re not over-riding them with local styling.

    What do you do when you want to include vector graphic objects, e.g. flow diagrams, mockups, etc?

    PS: have a cigarette? oh FFS..

  2. Alistair says:

    “you have to be constantly vigilant that you’re not over-riding them with local styling”

    Arrrgh! I hate that kind of rubbish. I’m not really surprised that it’s possible with Word. They’ve been piling in so many features for years, it can probably cook your dinner for you, in 60pt Papyrus with a purple background. Yum.

    Vector graphics I have not tried, but PNGs or JPGs are ok. I used this setup in earnest yesterday and it works very nicely indeed. I had a few diagrams and screenshots. Came out nice so it did.


©2010 Alistair Robinson