Scraps: Back Up Your Brain (in Emacs)

I’ve long missed Raymond Lowe’s wonderful program for MS-DOS called ’Scraps’. It implemented the idea that he called ’back up your brain’ and the point was to capture snippets, or ’scraps’ of information in a text-only database that was easy to search.

He wrapped some things around it like a calendar, to-do list, alarms, etc., but the real point was to capture things for easy recall later on, even years later. It was a great PIM (personal information manager) for its time.

But Y2K came along, and many Scraps features quit working. The database part still functioned, but there was another problem: the database was quite fragile, and in particular didn’t scale to large collections of scraps. When the database reached a megabyte or so (large in those days) you ran the risk of breakage and possible data loss.

I contacted Mr. Lowe but various restrictions prevented him from releasing the source code, so Scraps pretty much went away. A pity.

The ’scraps’ idea exists today in various forms. Google Keep, Evernote, Onenote, and others implement it in various ways. But this is about Emacs, right?

Long ago I used the Emacs database, EDB, to implement a sort-of scraps system. It worked but it wasn’t really the same. It kept everything in a single monolithic not-quite-text EDB database file. It seemed to scale to sizes I used back then, but that was then and this is now.

Along came ’deft’ a couple of years ago (2015, I think). ’deft’ implemented the scraps philosophy in a directory of small text files, each of which contained a single ’scrap’ (although it wasn’t called that). The only thing is, ’deft’ doesn’t scale to thousands of scraps representing tens of megabytes of text; it bogs down and becomes very slow, almost unusably so. Still, it was clever and implemented some rather good ideas.

To make a long story short I’ve written my own, new version of ’scraps’ for Emacs. It has these features:

• Scales to very large collections of thousands of scraps and tens of megabytes of text, while still running relatively quickly.
• Incorporates both regular expression searches and word (keyword) searches, with Boolean ’and’ facilities.
• Includes web clipping from internal Emacs browsers like w3m and eww (see also my org mode web clipper for a different option).
• Provides an Emacs replacement for ’deft’, Evernote, etc., at least up to a point.

I was a big Evernote user but I didn’t like their price increases, so I migrated my entire Evernote collection to Emacs ’scraps’ and it is working well for me.

’scraps.el’ is too long to post inline, but you can always get the latest version here:

http://www.bobnewell.net/filez/scraps.zip .

The current version is 0.70a. (This could be out of date; download to get the latest.)

The source archive contains extensive usage information and documentation in the form of a detailed manual.

Do let me know what you think: scraps@bobnewell.net .

Perhaps you too might be migrating from Evernote to Emacs scraps, even though scraps really doesn’t save graphics. Maybe you just want to stay within the Emacs ecosystem, a noble goal indeed.

There’s a migration routine to get stuff from Evernote; you can find it on the web and I won’t repeat it here (look for “How to Jump Ship from Evernote”). You’ll end up with a large number of text files with a .txt suffix. Here are a couple of things you might want to do.

Here’s something that’s completely experimental at this point; a means to do web clipping from a Chrome browser directly to an Emacs scrap. If there is a current selection in the web page, the selection is clipped, otherwise the whole page is clipped.

Now, there are some limitations:

• Only works (for now) with Chrome on a Linux system.
• Clipboard coordination must be enabled (see the comments in scraps documentation).
• Doesn’t do graphics, and some frames, links, etc., don’t come out correctly. (Again, it’s all about text.)
• May have local dependencies — I can only test on my own systems, not yours!

The method relies on ’xdotool’ and ’xclip’ (easily installed from your Linux repo) and the bash script below. Be sure to read through the script and make changes as required. (The script is now included in the package archive.)

#!/bin/bash

# Make a new Emacs 'scrap' from a full web page, or current selection,
# displayed in Chrome, using 'xdotool' and 'xclip'. Linux only.

# This code is part of Scraps for Emacs and is released under the same
# terms and licensing as scraps.el.

# THIS IS A WORK IN PROGRESS. There is a lot of tricky stuff here,
# including timing issues. Some web pages simply don't and won't and
# can't copy properly to text. Graphics, frames, some links, etc., can
# all fail to copy or copy imperfectly. On web pages that redefine
# keys such as Ctrl+c (like many Google app pages), there can be
# problems.

# SOME ASSEMBLY REQUIRED. PLEASE READ EVERYTHING.

# Be sure you have 'xdotool' and 'xclip' installed. Then put a
# shortcut to this script, which must have execute permissions, on
# your taskbar panel. When you find a web page you'd like to make a
# text scrap from, click the icon on the taskbar panel which
# represents this script. Usually it works. But not always. Be sure
# the page is fully loaded before you try to copy it!

# In Emacs the variable select-enable-clipboard must be t.

# ONE instance of Emacs must be running.  ONE instance of Chrome must
# be running.

# This code is somewhat system specific, so you might have to change a
# few things like window names, etc.

# Find the Chrome window. Error if none or multiple. Your naming
# convention may be different so modify to suit.
BWIDC=`xdotool search --name "google chrome" | wc -l`
if [ $BWIDC -ne 1 ] ; then
    xmessage -button "Not OK" "Either no Chrome window or multiple Chrome windows"
    exit 1
fi
BWID=`xdotool search --name "google chrome" | head -1`

# Find Emacs. If your naming convention is different you'll have to
# modify below. Error if no Emacs or multiple Emacs.
host=`hostname`
EWIDC=`xdotool search --name "emacs@$host" | wc -l`
if [ $EWIDC -ne 1 ] ; then
    xmessage -button "Not OK" "Either no Emacs window or multiple Emacs windows"
    exit 1
fi
EWID=`xdotool search --name "emacs@$host" | head -1`

# Go to the Chrome window. Sending keys to the window without changing
# focus doesn't seem to work.
xdotool windowactivate --sync $BWID

# Clear previous selection and check for clip.
#echo "" | xclip -i
#xdotool key --delay 25 Control+c
cchoice=`xsel`

# Go to the URL bar, and copy it. (Auto-select doesn't seem to work.)
xdotool key --delay 25 Control+l Control+a Control+c
sleep 1s

# Now go to Emacs.
xdotool windowactivate --sync $EWID

# Start a new scrap in Emacs.
xdotool key --delay 25 Control+c s n

# Enter saved URL as title of scrap.
xdotool key --delay 25 Control+y Return
# Put in the link. May be the same as the URL, depending on possible
# special characters cleaned up within scraps.el.
xdotool key --delay 25 Control+y Return

# Go back to browser.
xdotool windowactivate --sync $BWID

# Go into content area of web page.  (If you don't display the
# bookmark bar, remove one of the 'F6' below.)
xdotool key --delay 25 F6 F6

# If there is no selection get the whole page.
if [ "$cchoice" == "" ]; then
  xdotool key --delay 25 Control+a Control+c
  sleep 1s
# Otherwise use the previously saved selection
else
  echo "$cchoice" | xclip -sel clip -i
fi

# Go back to Emacs.
xdotool windowactivate --sync $EWID

# Paste content into scrap, and save.
xdotool key --delay 25 Control+y Control+x Control+s

# One last return to browser.
xdotool windowactivate --sync $BWID

# Deselect everything, if necessary.
if [ "$cchoice" == "" ]; then
  xdotool key Control+Shift+Home
fi

# And that's all.

If you try this please do let me know how it works out.