Making medieval manuscript transcription less painful with GNU Emacs

Introduction

I don't usually bother trying to introduce automations into my workflow if all I stand to save is a little time. Even if a certain common activity is taking a too long and automation would help, I usually stop at the question: how many seconds/minutes per day could actually be saved by automating, especially once the time taken to find or build a solution to the problem is factored in? When it comes to comfort, on the other hand, my philosophy is very different. If any frequent action you perform on your computer causes you physical discomfort (or even risk of injury), no amount of time spent tweaking things to find a suitable solution is "too much."

As a medievalist, one of my most common activities - sometimes to the tune of hours a day - involves transcribing digital facsimiles of medieval manuscripts so that I can compare them with one another. I do this because what we usually imagine as a single medieval literary work often survives in several different manuscript copies, each of which would have been produced by one or more scribes undertaking the laborious process of copying out the entire text by hand. As a result, the various surviving copies of any given story typically differ from one another in intriguing ways, whether because scribes intervened to change them on purpose, or simply because messages tend to get confused when people communicate them to one another in a chain (one need only consider the 'game of telephone' to see how this happens). The sense of discovery is exhilarating: with enough patience, one has the chance to find some truly crazy variation in these texts.

The problem is that like any long-term typing activity, transcribing can put a lot of strain on the hands and wrists. The issue is especially noticeable when dealing with a passage like this one, which comes from a manuscript of the Old French Roman de Renart:

The little squiggles that look like accent marks or apostrophes are abbreviations, and it is up to the reader to know how to make the words comprehensible by expanding them into their full forms. Because this is ultimately an act of interpretation (sometimes abbreviations are a little unclear), I like to mark some kind of visual difference in my transcriptions between the "real letters" actually written by the scribe and the extra letters that I believe the scribe intended to express in the form of abbreviations. To do this, I expand the abbreviations within parentheses: for example, if I were to transcribe this passage, the "q" near the middle of the first line would expand to "q(ue)".

This means that my right hand needs to contort to type parentheses constantly, multiple times per line, at a rate of many hundreds of lines per day. This is far from ideal: because the standard US keyboard layout requires chorded input (Shift+9 or Shift+0) to type parentheses, using parentheses all the time can quickly lead to wrist pain and increase the risk of repetitive strain injury (RSI).

Early approaches

It suddenly occurred to me: this seems like a problem that could be solved by writing a custom function for GNU Emacs, the famously extensible/re-programmable text editor in which I do almost all of my work. What if I could partially automate the insertion of the most common abbreviation forms that I find in the manuscripts I read?

My first attempt at writing such a function was dead simple:

12345(defun my/insert-standard-abbrev ()
  (interactive)
  (let* ((choices '("b(ie)n" "v(ost)re" "v(os)" "p(uis)" "m(ou)lt"))
         (selection (ido-completing-read "What to insert? " choices)))
    (insert selection)))

In case anyone reading this is unfamiliar with Emacs Lisp (I myself am only just learning), here's a quick breakdown: the function takes input from the user (the string selection), then compares it as it is being typed against a list of possible options (choices). Thanks to ido-completing-read, the whole word does not need to be typed, only enough to distinguish it from others in the list (more info at Xah Emacs Site). Then, by pressing Enter, the user can insert the string into the file at point (i.e. the cursor's current location the buffer currently being edited in Emacs). Note the use of let*, which differs from let by allowing a declared variable to reference a preceding variable. This is needed because selection references choices (see the relevant section of the Emacs Lisp Manual).

However, I quickly realized the deal-breaker with this was that I'd still have to be typing parentheses, defeating the whole point. This led to a second stab at the function, this time with more words incorporated as well:

1234567891011121314151617181920212223(defun my/insert-standard-abbrev ()
  (interactive)
  (let* ((choices '("bien" "vostre" "vos" "puis" "moult" "por" "et" "qui" "que" "certes" "con" "uostre" "nos" "plus" "par" "quant" "grant" "nostre"))
         (selection (ido-completing-read "What to insert? " choices)))
    (cond ((string= selection "bien") (insert "b(ie)n "))
          ((string= selection "vostre") (insert "v(ost)re "))
          ((string= selection "vos") (insert "v(os) "))
          ((string= selection "puis") (insert "p(uis) "))
          ((string= selection "moult") (insert "m(ou)lt "))
          ((string= selection "por") (insert "p(or) "))
          ((string= selection "et") (insert "(et) "))
          ((string= selection "qui") (insert "q(ui) "))
          ((string= selection "que") (insert "q(ue) "))
          ((string= selection "certes") (insert "c(er)tes "))
          ((string= selection "con") (insert "(con) "))
          ((string= selection "vostre") (insert "u(ost)re "))
          ((string= selection "nos") (insert "n(os) "))
          ((string= selection "plus") (insert "pl(us) "))
          ((string= selection "par") (insert "p(ar) "))
          ((string= selection "quant") (insert "q(ua)nt "))
          ((string= selection "grant") (insert "g(ra)nt " ))
          ((string= selection "nostre") (insert "n(ost)re "))
          (t (message "Invalid choice.")))))

This seemed like good progress: by using a conditional (cond) structure to take easy-to-type words and output them in the appropriate abbreviated form, the function was able to insert properly formatted abbreviations without the user's hands needing to leave the home row, and also without requiring the entire word to be typed. To save an extra keystroke, I added a trailing space to all of the outputs.

The last step was to bind it to a key sequence (on keys left for the user to bind without causing conflicts, see the wonderfully detailed Emacs manual):

1(global-set-key (kbd "C-c j") 'my/insert-standard-abbrev)

Redesign: writing a minor mode

It worked, but I had the sense more could be done. Even though my right hand no longer needed to stretch to reach parentheses, this new approach still involved an awful lot of typing, and I wasn't sure it was really helping all that much. It would be nice to have a way to call up the words directly by their first letter (in other words, without pressing c and j beforehand). The problem was that I would need to do this without stepping on the toes of any other key bindings in my configuration, which is now quite extensive.

Emacs has a solution: minor modes,A tip for anyone else thinking of writing their first minor mode: at this stage, I ran into a persistent problem in which the mode would be loaded globally for all buffers every time. Turns out I needed to read the manual more closely: don't forget that nil! which can change key bindings only in certain contexts (i.e., when they're enabled). By following a couple of greathttps://nullprogram.com/blog/2013/02/06/ guides,https://zerokspot.com/weblog/2019/07/07/defining-a-new-minor-mode-in-emacs/ I began work on my first minor mode, scribe-mode:

12345678(define-minor-mode scribe-mode
  nil
  :lighter " scribe"
  :keymap (let ((map (make-sparse-keymap)))
            (define-key map (kbd "C-c b") (lambda () (interactive) (insert "b(ie)n ")))
            (define-key map (kbd "C-c m") (lambda () (interactive) (insert "m(ou)lt ")))
            (define-key map (kbd "C-c e") (lambda () (interactive) (insert "(et) ")))
            (define-key map (kbd "C-c g") (lambda () (interactive) (insert "g(ra)nt ")))

Now by pressing Ctrl+c and then b, I can insert the text "b(ie)n". Note that I have Ctrl globally remapped to Caps Lock, which makes this a relatively easy move.

What about other letters like p, though, which might indicate the start of one of several different potential abbreviated forms? For these, I wrote some simple helper functions based on the earlier idea:

123456789(defun scribe-write-p ()
  (interactive)
  (let* ((choices '("puis" "por" "plus" "par"))
         (selection (ido-completing-read "Choice: " choices)))
    (cond ((string= selection "puis") (insert "p(uis) "))
          ((string= selection "por") (insert "p(or) "))
          ((string= selection "plus") (insert "pl(us) "))
          ((string= selection "par") (insert "p(ar) "))
          (t (message "Invalid choice.")))))

In addition to these, I added one extra function for inserting abbreviations in the middle of word. To allow for maximum flexibility, the user's input is not compared against a list of options.

1234(defun scribe-write-midword-abbrev ()
  (interactive)
  (let ((selection (read-string "What to insert? ")))
    (insert (concat "(" selection ")"))))

Here's what the minor mode's key map looked like at this stage. Letters that point to only one possibility have their associated words inserted right away, whereas letters that open onto several different possibilities offer a menu of options:

12345678910111213141516171819(define-minor-mode scribe-mode
  :lighter " scribe"
  :keymap (let ((map (make-sparse-keymap)))
            (define-key map (kbd "C-c b") (lambda () (interactive) (insert "b(ie)n ")))
            (define-key map (kbd "C-j b") (lambda () (interactive) (insert "b(ie)n ")))
            (define-key map (kbd "C-j m") (lambda () (interactive) (insert "m(ou)lt ")))
            (define-key map (kbd "C-j e") (lambda () (interactive) (insert "(et) ")))
            (define-key map (kbd "C-j g") (lambda () (interactive) (insert "g(ra)nt ")))
            (define-key map (kbd "C-j s") (lambda () (interactive) (insert "s(on)t ")))
            (define-key map (kbd "C-j v") 'scribe-write-v)
            (define-key map (kbd "C-j u") 'scribe-write-u)
            (define-key map (kbd "C-j n") 'scribe-write-n)
            (define-key map (kbd "C-j p") 'scribe-write-p)
            (define-key map (kbd "C-j q") 'scribe-write-q)
            (define-key map (kbd "C-j c") 'scribe-write-c)
            (define-key map (kbd "C-j r") 'scribe-write-renart)
            (define-key map (kbd "M-k") 'scribe-write-midword-abbrev)
            (define-key map (kbd "M-a") (lambda () (interactive) (insert "(n)")))
              map))

Improving the experience

After trying it out for a few days, though, I began to find that key bindings like C-j b weren't really a great solution to the problem of strain: they simply shifted the stress over to the left hand, which needed to reach over to Caps Lock all the time. What to do?

My Model M keyboard doesn't have a hyper or a super button, but it has one advantage: the right Alt key sits all by itself just below the palm of the right hand when one's fingers rest on the home keys. To press Alt (META or M in Emacs lingo) while typing, all one needs to do is lower one's right hand very slightly. It is therefore an ideal candidate for a button to press over and over for hours at a time. Since the effects of minor modes are confined to the buffer(s) in which they're active, there's no need to be shy about wreaking havoc on the normal Emacs key bindings by simply using M for everything - provided, of course, that you don't write over the key binding for a function that you would want to use while doing whatever work your minor mode is hoping to help with.

Here's the full code that I wound up settling on for scribe-mode.el:

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485(setq scribe-mode-map
      (let ((map (make-sparse-keymap)))
        (define-key map (kbd "M-b") (lambda () (interactive) (insert "b(ie)n ")))
        (define-key map (kbd "M-m") (lambda () (interactive) (insert "m(ou)lt ")))
        (define-key map (kbd "M-e") (lambda () (interactive) (insert "(et) ")))
        (define-key map (kbd "M-g") (lambda () (interactive) (insert "g(ra)nt ")))
        (define-key map (kbd "M-s") (lambda () (interactive) (insert "s(on)t ")))
        (define-key map (kbd "M-v") 'scribe-write-v)
        (define-key map (kbd "M-u") 'scribe-write-u)
        (define-key map (kbd "M-n") 'scribe-write-n)
        (define-key map (kbd "M-p") 'scribe-write-p)
        (define-key map (kbd "M-q") 'scribe-write-q)
        (define-key map (kbd "M-c") 'scribe-write-c)
        (define-key map (kbd "M-r") 'scribe-write-renart)
        (define-key map (kbd "M-k") 'scribe-write-midword-abbrev)
        (define-key map (kbd "M-a") (lambda () (interactive) (insert "(n)")))
        map))

(define-minor-mode scribe-mode
  nil
  :keymap scribe-mode-map
  :lighter " scribe")

(defun scribe-write-v ()
  (interactive)
  (let* ((choices '("vos" "vostre" "vous"))
         (selection (ido-completing-read "Choice: " choices)))
    (cond ((string= selection "vostre") (insert "v(ost)re "))
          ((string= selection "vos") (insert "v(os) "))
          ((string= selection "vous") (insert "vo(us) "))
          (t (message "Invalid choice.")))))
(defun scribe-write-u ()
  (interactive)
  (let* ((choices '("uos" "uostre" "uous"))
         (selection (ido-completing-read "Choice: " choices)))
    (cond ((string= selection "uostre") (insert "u(ost)re "))
          ((string= selection "uos") (insert "u(os) "))
          ((string= selection "uous") (insert "uo(us) "))
          (t (message "Invalid choice.")))))
(defun scribe-write-n ()
  (interactive)
  (let* ((choices '("nos" "nostre" "nos"))
         (selection (ido-completing-read "Choice: " choices)))
    (cond ((string= selection "nostre") (insert "n(ost)re "))
          ((string= selection "nos") (insert "n(os) "))
          ((string= selection "nous") (insert "no(us) "))
          (t (message "Invalid choice.")))))
(defun scribe-write-p ()
  (interactive)
  (let* ((choices '("puis" "por" "plus" "par"))
         (selection (ido-completing-read "Choice: " choices)))
    (cond ((string= selection "puis") (insert "p(uis) "))
          ((string= selection "por") (insert "p(or) "))
          ((string= selection "plus") (insert "pl(us) "))
          ((string= selection "par") (insert "p(ar) "))
          (t (message "Invalid choice.")))))
(defun scribe-write-q ()
  (interactive)
  (let* ((choices '("qui" "que" "quant"))
         (selection (ido-completing-read "Choice: " choices)))
    (cond ((string= selection "qui") (insert "q(ui) "))
          ((string= selection "que") (insert "q(ue) "))
          ((string= selection "quant") (insert "q(ua)nt "))
          (t (message "Invalid choice.")))))
(defun scribe-write-c ()
  (interactive)
  (let* ((choices '("certes" "con"))
         (selection (ido-completing-read "Choice: " choices)))
    (cond ((string= selection "certes") (insert "c(er)tes "))
          ((string= selection "con") (insert "(con) "))
          (t (message "Invalid choice.")))))
(defun scribe-write-renart ()
  (interactive)
  (let* ((choices '("Renart" "Renars"))
         (selection (ido-completing-read "Which declension? " choices)))
    (cond ((string= selection "Renart") (insert "R(enart) "))
          ((string= selection "Renars") (insert "R(enars) "))
          (t (message "Invalid choice.")))))
(defun scribe-write-midword-abbrev ()
  (interactive)
  (let ((selection (read-string "What to insert? ")))
    (insert (concat "(" selection ")"))))

(add-hook 'nxml-mode-hook 'scribe-mode)
(provide 'scribe-mode)

The hook at the end tells Emacs to load this minor mode whenever I open an XML file, which is helpful because all of my transcriptions follow a (very) simplified form of the TEI guidelines.

One last step: this line in my Emacs configuration file tells the program to load scribe-mode.el (note that this assumes scribe-mode.el is in the Load Path):

1(require 'scribe-mode)

Here's a look at the process in action (image generated with gif-screencast.el and keycast.el):

Wrapping up

And there you have it: no more pain, and a fun excuse to learn some more Emacs Lisp!

I think this is a great example of why Emacs is so amazing: it puts the user in a direct, active relationship with their working environment. Even though it seems like just another text editor at first, having an immediately useful malleable system at your fingertips every day has a certain cumulative effect. If you use Emacs all the time, it won't be long before you start getting curious about how you might be able to make your life a little easier - or make your wrists hurt a little less! - by molding the program to your exact needs.

Comments welcome by e-mail; see discussions about this post on Reddit and lobste.rs.