Since R is open-source, it is HIDEOUSLY time-consuming to learn independently. Having installed the software one then downloads various PDFs offering Introductions to R, works doggedly through the PDFs, blunders cluelessly through the maze of FAQs and Help, buys and works doggedly through books such as Data Analysis and Graphics Using R (Maindonald and Braun) and R Graphics (Paul Murrell), occasionally sending apologetic pleas for enlightenment by e-mail to those in the know. Statisticians tend to feel that both statistics and statisticians are strangely underrepresented in fiction, so very distinguished statisticians are startlingly helpful, but still one is blundering about -- all this while trying to deal with people in the movie biz, the book biz, any other biz. So the project kept getting disrupted, and each time it got disrupted it was necessary to go back to the beginning and start from scratch. In September 2005 I applied for a Guggenheim Fellowship for the project, in March 2006 I was told I'd got it, but same old same old same old, many wonderful people thought I had better things to do with my time and so it goes.
Here are a few graphics from R from one of my many PDFs:
Awwwwwwww.What I mean is, although I did pitch the project to the Guggenheim as a way of permitting fiction to show us the structures of power in the world -- the world of finance, which permeates all lives, has been transformed in the last 20 years by the development of financial instruments such as options, futures and derivatives, which could (but don't) transform our understanding of chance, of risk, in daily life -- I really just wanted an excuse to write a book using the sort of graphics I had come across in Jim Pitman's probability, which I found utterly delectable. The R Gallery offers many more examples of delectability. My response to these graphics is like an illustration of Ayer's yah-boo emotivist theory of value, it's like the average Stern-reader's response to dear little Knut (and dear unjustly neglected little Ernst): Awwwwwwww. Awwwwwwwww. Awwwwwwwww.
What can I say? I LIKE a page of tiny histograms. I LIKE Tuftean sparklines (not that you can do them in R, but this is another problem.) I LIKE -- no, to be perfectly honest, I ADORE a panel of tiny boxplots. Awwwwwwwwwwww.
Well. Poor old head. Poor old head.
Two and a half years ago I had a lot of problems with an agent and a film director. The film director was "passionate" about making a film of The Last Samurai but had been thwarted in his efforts to proceed because, um, it is impossible to locate a writer on the planet if the writer has not taken the precaution of setting up a website. Instead of doing what Kurosawa would have done (find the author and camp out on her doorstep) he had chosen to talk passionately about the book to his agent at William Morris and to other agents at William Morris. An agent at WM to whom he had talked passionately got in touch with me after I got in touch with them, offered to represent me, started wheeling and dealing, turned out to know nothing about the publication history of The Last Samurai because, um, the writer had not taken the precaution of setting up a website.
Bad news, bad news, bad news, I thought I had better set up a website. A friend had a friend. The friend's friend set up a website, yes, but she liked Flash so she set it up in Flash. She thought text was boring, converting Word documents to HTML was unfun, so she put all the reviews and text on the site as downloads, all surfers' pet hate.
OK.
Things went wrong with the agent and the director, another story. The website has been there for two and a half years, and because it was too hi-tech for its owner it has been dead. This blog, of course, is very lo-tech. The blog has been going for just under 4 months, and it has had over 150 posts, and it has a shockingly cluttered sidebar that will only get worse -- in fact, it has all kinds of material that would really work much better on a website, but it all goes on the blog because the blog is lo-tech and requires no social interaction, no negotiation, no give-and-take as a prerequisite to posting.
OK.
Getting a Guggenheim Fellowship -- getting any fellowship, for that matter -- is quite time-consuming. My application took a month to write. One has to write up the project, write up a CV, persuade three or four distinguished people to take time away from their own work to write in support of the project. It's tiring, it's time-consuming, but it's straightforward: you are trying to convince your peers that you will write a work of genius.
Getting a book published is quite a different matter. If the book is doing something new, especially if it is doing something which falls foul of cultural prejudice (most novel readers DO NOT like mathematics), a lot of stage-setting needs to be done to persuade agents and publishers that it should be shown to the public. Something needs to be done to -- well, this is actually quite tricky. It would be good to do something to persuade the time-pressed sceptic that thousands of readers could see a page of tiny histograms and say AWWWWWWW. But it would also be good to give this time-pressed sceptic some way of seeing how the book is radically new, important... Point is, this is a much more challenging rhetorical exercise than persuading distinguished writers and critics that you will write a work of genius. THAT took a month. THIS is something you might need to refine for six months, a year, maybe as long as it takes to write the actual book. So you need a very lo-tech website that you can keep going back to. (If you are ALSO trying to persuade both publishers and the reading public that they would like to learn Arabic... let's just say that this too needs a lot of groundwork.)
Now.
[HOW long this is.]
If someone has put a lot of work into a project, as my last web designer did into the website, it is socially awkward to take him or her off the project. If this can be done quite simply and cleanly -- if all the images and files are on the server -- it may be possible to do it without a lot of friction. If it is necessary to ask that person to provide a CD with all the files, or to upload files to the server, it will take an enormous amount of social savoir faire, an enormous amount of emotional energy. BUT dealing with agents and publishers takes up such a staggering amount of social labour, there is really nothing left for this kind of task. If the files are not on the server available for use, practically speaking everything will have to be done from scratch, all the images and files will have to be rounded up again. BUT this is a lot of work, something that is really not compatible with getting to grips with R AND writing a book. If that is what's required, moving the website will have to wait -- UNLESS, of course, one can pay someone to do the drudgery of rounding up all the material.
BUT.
If you have a simple, well-defined little task, the number of people who have the relevant skills AND can do the task without a lot of oversight is very large. But if you have a task that involves doing everything from scratch, the number of people who have both the skills AND the patience to slog through, bringing the necessary attention to detail to bear, is very small. So you are not really divesting yourself of drudgery, you are taking on both a recruitment problem and a management problem. Both of these take up so much mental energy you are almost certainly better off doing the job yourself.
So, anyway. I hoped that what I had was a clean, simple little job: taking the files on the server and converting the hi-tech dead website into a lo-tech website that would see as much action as this lo-tech blog. But I didn't. My former web designer had embedded all the files in a Flash movie, so everything would have to be done from scratch. The friend who had offered to haul the website into Dreamweaver had roughly the tedium threshold of the original designer. So this was bad. The hi-tech Flash site was converted into a hi-tech Dreamweaver site, a site using CSS. But I don't want to clutter up my mind with CSS because I HAVE to get up to speed on R; the opportunity cost of wading through my Dreamweaver manual, sussing out CSS, is, of course, the amount of progress I could have made in the time with Lemon's Kickstarting R, Murrell's R Graphics usw. Poor old head. Poor old head.
I then get some e-mails from extremely fabulous Mr Ilya, the not-very-noticeably-abject Other of the Bauhaus functionalism embodied by R. I get an envelope in the post from EFMI which is one solid mass of duct tape apart from two small apertures in the duct tape where two addresses, mine and his, appear amidst the riot of adhesive. I don't really like to OPEN this thing of joy, but when I take a knife to it out come: 1. a DVD of of TELEVISION VIRTUAL FIREPLACE. Create the sensation of instant warmth! [what precisely the force of 'sensation' is in this context, of which we are told, LOOKS AND SOUNDS JUST LIKE THE REAL THING -- but Awwwwwwwwwww] Amaze your friends! Impress your loved ones! TURN YOUR TV INTO A REALISTIC FIREPLACE! and 2. several photocopies of drawings by Goya. Back in the days when EFMI was here we used to go to Sarotti, which has faux flames on an LCD behind the bar, so Awwwwwwwwwwww. And I get another e-mail about chlamydia sweeping through the koala population. And ANOTHER e-mail about EFMI hassling John Howard about numerous scandals, spreading annoyance and gene--
The friend who hauled the website into Dreamweaver has been quite unbelievably helpful and kind. I think she literally saved my life a few months ago ('literally' as in if not for this friend people would have been reading my obituary), and she has fought with the telecommunications bureaucracy on my behalf, and she is very funny and smart, and when I thought I had a simple, clean little job to do I thought I should give it to her. But when a job is not simple and clean, when a job requires the equivalent of a Guggenheim application to explain, it is very dangerous to hand it over to someone, because it will be impossible to get on with any other work -- and when the dust has settled there will be a lot of work left to be done. And as Mies said, Die Seele ist in den Details. Poor head, poor head, poor head, poor crazy head.
I think Kickstarting R is new. It says: Kickstarting R was initially compiled to help new users by requesting accounts of "... things that drove you crazy the first time you used R". Awwwwwwwwwww.
6 comments:
Well, to concentrate only on the very shining bright part of this post, I too find the histograms quite delectable; of course your delectability levels have to be extremely high for it to be worth the trouble of writing the novel! It sounds so great, though! Other stuff maddening...
And to concentrate only on the mechanical part of the post, I wonder if you have looked into a system like drupal or joomla for the web site? On the off chance tat you haven't, these content management systems are built for community sites, and once you have it going you can make changes from a browser (no ftp-ing stuff here and there). So you could give access to a small (or large) number of people and they could do different bits of the job. Just a thought.
I understand I think, about little tasks and starting from scratch. In the winter before the war, it started raining in Hebron as I was TRYING to calibrate these four reservists so we could begin our training. The reservists were there for one day of training, the weapons were later to be returned so the reservists can drive down south I think and get new weapons and calibrate them again so they can begin guarding. The army works in mysterious ways. Also, the last time the reservisst fired a sniping weapon was probably over 7 years ago, and because it was (as I said) raining, the calibrating papers that help you calibrate the weapons all flew into a puddle and were luckily all wet right below the point to which the sniper had to aim to calibrate his weapon at 25 meters, but UNfortunatly not below where the bullet should eventuelly hit at 25 meters, because of the ballistic arch. (I would write exact numbers but then you could guess the gun and the bullets, the unit, big brother etc etc) So. The reservists COULD aim just as they would anyways, but I couldn’t with accuracy move the aims to the exact point I needed them to be because it was messed up from the rain, and bringing more calibrating papers would mean walking for 75 minutes and delaying the training. However, ABOVE the point where the snipers had to aim were marks of other weapons, designed for their ballistic arches destined hits at 25 meters. What I thought I could do is tell the snipers to aim as they always do, then move the aims to the calibrating point of another random weapon (at the 25 meters) which would create a completely new ballistic arch!!! As the reservist were filling out bullets I could calculate that NEW ballistic arch with the help of numbers and experimentation to be sure, so that for our training if we shoot for the 275 I could tell them what the deviation was, if we were shooting to the 350 I could tell them what the deviation was---what is the difference, for training purposes if the deviation is 6 cm up or 12 down? The reservists sure didn’t remember, and this was a brand new ballistic arch! At the end of the training I would need to explain…No, just tell them every single deviation I said would be wrong for any weapons other than these, hand them a sheet with the right deviations for the weapons they’ll be calibrating down south, and Wallla! The whole thing ran so smoothly all I could do was thank the rain for giving me SOMETHING to do with my brain. We were training at shooting to the 150 while sitting when I was called to ambush the eastern gate for some odd threat or another, and another tall boy came to take over the training. I did not know what to do. You would think, the boy was a sniper, and that maybe YOU have no idea what I just described doing here, but with a few minutes of explaining and some patience and a cute smile surely I could explain to the boy what to do. You could think that, and you would be wrong.
Jenny -- I once read Michael Caine talking about working with Olivier on Sleuth. He said Olivier was somehow only able to get a handle on a part if he had a bit of business or a bit of costume or disguise. In this case, I think, Olivier came up with a fake moustache and suddenly hit his stride with the part. Writing fiction is like that for me, it's really only when I find something gimmicky to do with the page that the story and characters take off. So it would be more trouble to write a novel without a slew of tiny histograms (or something like them).
Tom. I am inexpressibly touched to have a reader who thinks I might have looked into a system like drupal or joomla -- and even thinks that there is just an off chance, rather than an overwhelming probability, that I haven't. I will have a look at these. That could be a good idea. I should say that quite a while back designer friend 2 suggested that I might want to set up a wiki (so she is really not a control freak and has consistently tried to be helpful). On the other hand, not to be too much the tired cynic, I do ask myself, given my record of disasters on the person-to-person level of management, whether there is any reason to expect anything better in setting up a group project.
leitmotifish, if Joel Spolsky were getting advice on gun calibration from his anonymous contributors rather than wild-eyed rants about Opera he would probably be less dismissive of comments as a contribution to a blog.
Oh, I just meant the trouble of writing ANY novel, not at all on the basis of histograms or otherwise! The delectable quality of the idea and the things you have to do along the way (for me, right now though I can't do it till I'm done with my other book, reading insane books about Lapland and Alfred Nobel) are the only things that make it tolerable, writing a novel and revising it into some fit state is such a horrible long job...
How different I am, Jenny. I find the delectable part are the sentences, though of course my pleasure in writing them doesn't mean they're delectable for anyone else to read. It's all the other stuff I mostly find so tedious, though maybe I just haven't found my histogram.
Post a Comment