When scoring classifiers one wants to reward both precision and recall. A simple and useful linear function of precision and recall is just 2 * TP / ( 2 * TP + FP + FN) , where TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives.

We can do better though. For some discussion, issues, and alternates see "Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation" (2011) by David M W Powers in the Journal of Machine Learning Technologies 2 (1): 37–63.

Two lines is all it takes for hacker's image and video gallery:

for f in *.jpg ; do echo "<a href=\"$f\"><img width=\"75%\" src=\"$f\"></a>" ; done > index.html for v in *.webm ; do echo "<a href=\"$v\">Click to download $v <video src=\"$v\" controls style=\"width=75%\"/></a>" ; done >> index.html

I found Zach Poff's Resistor Identification PDF to be very helpful, but the SMD and +/-1% tolerance sections were not very useful to me when sorting a bag of cheap resistors, so I made a Hobbyist Resistor Identification Codes PDF (from an SVG) in Inkscape of something easy to print out.

Kōans are universal Computer Science folklore. Especially the AI ones.

Recursion is a fundamental structure. Lisp-in-lisp is seen as a great achievement, as is when any programming language becomes self-hosting.

I had never connected these very deeply, possibly because I didn't know the first thing about kōans:

"...in the beginning a monk first thinks a kōan is an inert object upon which to focus attention; after a long period of consecutive repetition, one realizes that the kōan is also a dynamic activity, the very activity of seeking an answer to the kōan." [1]

I am actually amazed I've used emacs for programming for so long without writing this; bind to Ctrl-, and wonder how you lived without it:

                (defun mtd-indent-after-comma-space ()
                  (interactive)
                  (re-search-forward ",[ ]+" (line-end-position) t)
                  (replace-match ",")
                  (newline-and-indent))
                (global-set-key (kbd "C-,") 'mtd-indent-after-comma-space)
              

This is the right use of python's powerful default kwargs feature:

                import time

                def report(when=lambda: time.time()):
                    print when()

                report()
                time.sleep(5)
                report()
              

The wrong one is documented all over, but often without a clear statement that "Default parameter values are evaluated when the function [is defined]"ref.

I think it's more useful to ask people to take away the lambda: and ask themselves why that doesn't work as they probably would expect.

I've made new RPMs of QuickFIX 1.13.3 for Fedora 16 that you can download; the patches are also available separately.

I'm working on getting QuickFIX officially into Fedora; if you're interested in helping please check redhat bugzilla #606421.

I've made new RPMs of QuickFIX 1.13.3 for Fedora 15 that you can download; the patches are also available separately.

Thanks to Dennis Fleurbaaij for some patches.

I'm working on getting QuickFIX officially into Fedora; if you're interested in helping please check redhat bugzilla #606421.

I backported tracker's "index & search for numbers" functionality (without its config-ui nicety) from tracker-bug #503366 to Fedora 14's tracker now-ancient 0.8.17 so I could search for numbers; the change was surprisingly small given how much tracker has moved on:

diff -ur tracker-0.8.17-p2/src/libtracker-fts/tracker-parser.c tracker-0.8.17-patch-index-numbers-gnome-bugzilla-num-503366/src/libtracker-fts/tracker-parser.c
--- tracker-0.8.17-p2/src/libtracker-fts/tracker-parser.c	2011-03-30 18:11:50.717630143 +0800
+++ tracker-0.8.17-patch-index-numbers-gnome-bugzilla-num-503366/src/libtracker-fts/tracker-parser.c	2011-03-31 00:48:06.160630144 +0800
@@ -354,8 +356,7 @@
 					}
 				}
 
-				if (!is_valid ||
-				    word_type == TRACKER_PARSER_WORD_NUM) {
+				if (!is_valid) {
 					word_type = TRACKER_PARSER_WORD_IGNORE;
 					is_valid = TRUE;
 					length = 0;
@@ -378,18 +379,9 @@
 		if (!start) {
 			start = g_utf8_offset_to_pointer (parser->cursor, char_count-1);
 
-			/* Valid words must start with an alpha or
-			 * underscore if we are filtering.
-			 */
-
-			if (type == TRACKER_PARSER_WORD_NUM) {
-				is_valid = FALSE;
-				continue;
-			} else {
-				if (type == TRACKER_PARSER_WORD_HYPHEN) {
-					is_valid = parser->parse_reserved_words;
-					continue;
-				}
+            if (type == TRACKER_PARSER_WORD_HYPHEN) {
+              is_valid = parser->parse_reserved_words;
+              continue;
 			}
 		}
 
@@ -459,7 +451,9 @@
 		return FALSE;
 	}
 
-	if (word_type == TRACKER_PARSER_WORD_ALPHA_NUM || word_type == TRACKER_PARSER_WORD_ALPHA) {
+	if (word_type == TRACKER_PARSER_WORD_ALPHA_NUM
+        || word_type == TRACKER_PARSER_WORD_ALPHA
+        || word_type == TRACKER_PARSER_WORD_NUM) {
 		gchar       *utf8;
 		gchar       *processed_word;

It's very nice to be able to search for numbers now.

I wanted to see what I could do with vecto so I had another go at Common Lisp on Fedora, almost three years since it crashed and burned for me. "yum install sbcl" was fine, but then hit the same symptom as years ago: libraries don't install using ASDF-INSTALL (this was the "old" way, even in in 2008).

Luckily, the other old symptom of "libraries don't install the new way, either" has changed: the old old way is gone, the old new way is now the old way, and the new way is really new, and actually works:

sudo yum -y install sbcl
git clone git://gitorious.org/clbuild2/clbuild2.git
cd clbuild2
./clbuild quickload vecto
    ...lots of downloading
./clbuild prepl --noinform
    ...much amusement with dumping core files and such
* (require 'vecto)
    much system loading
NIL
* 

If things keep working this well soon people will be asking heretical things like "stop telling me about system loading when it works".

To convert scanned PDFs to text on linux - where the PDFs don't already have text embedded in them, of course - I have come up with the below process using tesseract for the OCR and unpaper to correct for the vagaries of the scanning process (slightly off-kilter document images, for example).

I've started with PNG files, but of course you can just use ImageMagick to convert foo.pdf foo.png to get n foo-i.png files for an n-page PDF.

 
for f in ${output_basename}*.png ; do
    stem=$(basename $f .png)
    pngtopnm ${stem}.png > ${stem}.pnm | tee -a ${output_basename}.log
    unpaper -b 0.5 -w 0.8 -l single -s a4 ${stem}.pnm ${stem}-unpapered.pnm | tee -a ${output_basename}.log
    convert ${stem}-unpapered.pnm -depth 8 -monochrome ${stem}.tif | tee -a ${output_basename}.log
    tesseract ${stem}.tif ${stem}-tesseract | tee -a ${output_basename}.log
done

If you're using Fedora linux, yum can install all the prerequisites for you:

 
sudo yum -y install ImageMagick tesseract poppler-utils unpaper netpbm-progs

When writing a Python extension module, the basics of using another python module's attribute are not overly documented. Constructing an instance of another module's class, for example, is a bit of a mystery. There are at least two object creation primitives and a lot of C APIs to get in the way of integrating gracefully with python-implemented code. Here's how one can do it:

• To import a module, normally one would want to have the same thing happen as when import modulename is interpreted. This is done by the PyImport_Import function, like this:

  PyObject *mod_name = PyString_FromString("modulename");
  PyObject *module = PyImport_Import(mod_name);
  Py_DECREF(mod_name);
  if (module == NULL)
  {
      Py_RETURN_NONE;
  }
                    

• Then, to get a module's attribute, like decimal.Decimal, recall that module (and class) attribute access (the ".") means the same thing as looking up the name in the module's dictionary. The general "get some attribute of this object" function is PyObject_GetAttrString, and it's used like this (continuing the code from above):

  PyObject *attribute = PyObject_GetAttrString(module, "attribute");
  if (attribute == NULL)
  {
      Py_DECREF(module);
      Py_RETURN_NONE;
  }
                    

• Call the object's constructor. Again, we want to do what the interpreter would do when confronted with module.attribute(); this is PyObject_CallObject:

  attribute_ctor_args = Py_BuildValue("(s)", value.c_str());
  PyObject *attribute_instance = PyObject_CallObject(attribute,
                    attribute_ctor_args);
  Py_DECREF(attribute_ctor_args);
  Py_DECREF(attribute)
  if (attribute == NULL)
  {
      Py_RETURN_NONE;
  }
  else
  {
      return attribute;
  }
                    

• We could pass arguments to the constructor, if we create them first - perhaps using Py_BuildValue:

  PyObject *attribute_ctor_args = Py_BuildValue("(s)", "any char * will do");
  PyObject *attribute_instance = PyObject_CallObject(attribute, attribute_ctor_args);
  Py_DECREF(attribute_ctor_args);
                    

Lisp reference sheet

Save/restore your MBR:

• without the partition table:

  dd if=/dev/hda of=hda-mbr-nopart bs=446 count=1

• with the partition table:

  dd if=/dev/hda of=hda-mbr-full bs=512 count=1

• just the partition table:

  dd if=/dev/hda of=hda-parttable bs=64 count=1 skip=446

I've got some patches and an updated specfile to get quickfix 1.13.3 built on Fedora 12 and higher. This includes the python bindings (tested a bit) and examples (untested).

You can download the rpms here.

I am going to work on getting them submitted to the quickfix project and into Fedora.

The absolute minimum you need to know about QuickFIX / FIX programs is:

• FIX specifies how processes talk to each other over a transport layer like TCP/IP.
• FIX clients talk (initiate message exchanges); FIX servers listen (finish message exchanges).
• FIX servers can do different things, like process orders or send market data to clients.
• The first thing you will usually write with FIX is a client. It will open a port to a FIX server and send a FIX login message.
• FIX messages are key/value pairs where the keys are numbers and the messages are sent over the wire as header and message data - the message data will be almost human-readable.
• A FIX message looks almost like this on the wire:
  E.....@.@.N{RDPl...]. .P $+..%%....6`...... {....]Q.8=FIX.4.4.9=82.35=5.34=6.49=XXXXXXXXXXXXXXXXXXXXXX.52=20100617-10:37:36.348.56=XXXX.57=XXXXXXXXX.10=249.
  Notice the "8=FIX..." part, which is the message / payload starting.

There are different versions of FIX...versions 4.1 to 4.4 are most prevalent, and v5.0 is the latest. They're not terribly compatible. And just because an app is FIX vX.Y compliant doesn't mean you'll be able to rely upon much of the content of key data fields...like any message format, the message contents a) matter; and b) are often abused by low-quality implementors.

Can you think of anythine else? Let me know via email.

Here's how I got quickfix running on Fedora 12:

                wget http://kojipkgs.fedoraproject.org/packages/quickfix/1.12.4/9.fc12/x86_64/quickfix-1.12.4-9.fc12.x86_64.rpm
                sudo yum --nogpgcheck localinstall quickfix-1.12.4-9.fc12.x86_64.rpm
              

Ok, so that's just to get libquickfix.so. To actually do something useful, like connect to a FIX endpoint, there's more. But before we get to that, how about building the latest version of quickfix:

#standard rpmdev fedora setup:
sudo yum -y install yum-utils rpmdevtools rpmdev-setuptree

# verify that we can build the existing RPM
sudo yum-builddep quickfix-1.12.4-9.fc12.src.rpm
rpm -Uvh quickfix-1.12.4-9.fc12.src.rpm
rpmbuild -ba ~/rpmdev/SPECS/quickfix.spec

#WORKSFORME.  now let's upgrade to 1.13.3:
sed -i -e 's/1.12.4/1.13.3/; s/9{%dist}/1{%dist}/' ~/rpmdev/SPECS/quickfix.spec
rpmbuild -ba ~/rpmdev/SPECS/quickfix.spec

              

Next update: getting python, ruby, and java bindings built.

There are a few guides to installing a Maemo 5 scratchbox on a Fedora 12 x86_64 machine, but none of them worked for me without modification.

In particular, the vm.mmap_min_addr magic really ate up a few minutes. To save one or two other people some time, here's what worked for me:

# do this as root

yum -y install Xephyr dbus hal
groupadd sbox
usermod -a -G sbox $LOGNAME
echo >> sysctl.conf <<EOF 

# for maemo scratchbox (n900) from http://talk.maemo.org/showthread.php?t=34924 
abi.vsyscall32=0

# avoid mmap problems installing .debs:
# http://lists.scratchbox.org/pipermail/scratchbox-users/2009-August/001530.html
vm.mmap_min_addr = 1048576
EOF

# increase apt cache memory limit, for example http://lena.franken.de/linux/debian_and_vserver/commands_to_install.html#NotEnoughMemory
echo "APT::Cache-Limit 50000000" >> /etc/apt/apt.conf

sysctl -p

# do this as non-root user

wget http://repository.maemo.org/stable/5.0/maemo-scratchbox-install_5.0.sh http://repository.maemo.org/stable/5.0/maemo-sdk-install_5.0.sh 
chmod a+x ./maemo-scratchbox-install_5.0.sh ./maemo-sdk-install_5.0.sh

patch maemo-scratchbox-install_5.0.sh <<EOF
--- maemo-scratchbox-install_5.0.sh~    2010-01-13 09:00:40.000000000 +0000
+++ maemo-scratchbox-install_5.0.sh     2010-02-03 02:33:22.000000000 +0000
@@ -328,6 +328,7 @@
                        ;;
                s)
                        __type=tgz
+                       __force=yes
                        __scratchbox=$OPTARG 
                        ;;
                g)
EOF

sudo ./maemo-scratchbox-install_5.0.sh -s /opt/scratchbox -u $LOGNAME

./maemo-sdk-install_5.0.sh -d -s /opt/scratchbox -m maemo-sdk-debug 

Transcribed + extended a bit from randystuph's blog entry:

(System/setProperty javax.naming.Context/INITIAL_CONTEXT_FACTORY
       "org.apache.naming.java.javaURLContextFactory")
(System/setProperty javax.naming.Context/URL_PKG_PREFIXES
       "org.apache.naming")
(def ctx (new javax.naming.InitialContext))

(doseq [node ["java:" "java:comp" "java:comp/env" "java:comp/env/jdbc"]]
       (.createSubcontext ctx node))

; Sybase example - SybConnectionPoolDataSource
(def sybase-datasource (new com.sybase.jdbc2.jdbc.SybConnectionPoolDataSource))
; non-pooled version: com.sybase.jdbc2.jdbc.SybDataSource
(doto sybase-datasource
    (.setDatabaseName "mydbname")
    (.setServerName "hostname.here.com")
    (.setPortNumber 1400)
    (.setUser "myuser")
    (.setPassword "password"))
(.bind ctx "java:comp/env/jdbc/sybaseds" sybase-datasource)

; JNDI lookups look like:
(.lookup ctx "java:comp/env/jdbc/sybaseds")

; Sybase connections look like:
(def sybase-connection-pool (.getPooledConnection sybase-datasource))
(def sybase-connection (.getConnection sybase-connection-pool))
; non-pooled version:
; (def sybase-connection (.getConnection sybase-datasource))


; Oracle example
(def oracle-datasource (new oracle.jdbc.pool.OracleConnectionPoolDataSource))
(doto oracle-datasource
    (.setURL "jdbc:oracle:thin:@host.name.here:porthere:schemahere")
    (.setUser "oracleuser")
    (.setPassword "oraclepasswd"))
(.bind ctx "java:comp/env/jdbc/oracleds" oracle-datasource)

; JNDI lookups look like:
(.lookup ctx "java:comp/env/jdbc/oracleds")

; Oracle-connections look like:
(def oracle-connection (.getConnection oracle-datasource))

              

I couldn't find any web page showing the Simplest LaTeX Document That Could Possibly Work, so here it is:

\documentclass{minimal} \begin{document} [This space intentionally left blank] \end{document}

I then found this "minimal" example.

Great paper: On Understanding Data Abstraction, Revisited.

To summarise: Object-oriented programming and abstract data types are two related but distinct forms of data abstraction. The difference between the two is akin to the difference between thinking in terms of characteristic functions and algebras.

The definitions of objects (specifically, objects referred to when one speaks about object-oriented programming) and abstract data types are

1. Nobody agrees on what "Object-oriented" means, but it's either:
   • The classical definition from Smalltalk
   • The author's conception of objects as something that has:
     1. an interface of functions from known types to known types
     2. autognostic only: can only access other objects through their public interfaces
2. Abstract data types have an agreed-upon meaning as something that has:?
   1. a public name
   2. a hidden representation
   3. operations (public) to create, combine, and observe values of the abstraction

If you want to write python and use it in Excel, check out PyXLL. Using some C++/CPython dark magic, you can call Python functions as Excel user-defined functions (UDF) - it's very cool. It's a hard technical bridge to span but it's been done elegantly and simply. You write python and you get excel functions. The software is in private beta but requests considered quickly, so it's easy to try. The guy who wrote it is great and sits next to me, so I know he wants feedback. Check it out.

...(and vice-versa) by Attila Szegedi:

[T]here are four reference types in Clojure, and they differ in three concurrency attributes, namely:

• synchronicity (whether the change-initiating thread waits for the change to complete)
• coordination (of multiple changes across multiple references - transactions, basically)
• and value sharing (among threads)

Vars: unshared (thread-local), synchronous (consequence of thread-locality), uncoordinated (consequence of thread-locality).

Atoms: shared, synchronous, uncoordinated. They're basically a compare-and-swap.

Agents: shared, asynchronous, uncoordinated. Change functions are queued, and each function is run by agent sequentially using a thread pool.

Refs: shared, synchronous, coordinated (transacted). They're the software transactional memory implementation.

Also worth noting is that agents are integrated with the STM system - any sends to an agent that are initiated from within a STM transaction will get delayed until the transaction commits (otherwise the transactions could have external side-effects via agent sends).

Shed-storm n.: bikeshedding on a mailing list that turns sour and degenerates into a flamewar.
synonyms: shed-war, shed-fest;
root: flame-war, flame-fest

Dengler's Law of Inexorable Software Communism is:

Because one can't (or won't) throw code away, one is guaranteed a job for life maintaining the code. The only disadvantage is: one has to maintain the code.

Software engineers are always thinking "argh! I wish I could just throw this all away and start again". I've seen the best software engineers go beyond thinking about it and actually do it time and again. But the mediocre ones tend to only think it. Because they're too scared. And they know it will take too long. Whereas the best just do it, because it doesn't take them long.

Some of you might be thinking "If they're so good, shouldn't they have done it right in the first place?". In fact, they did do it right in the first place. But then something changed to make it not the best solution, and that something was fundamental enough a change to require a re-write.

Williamson's First Corollary to this is:

If one has the forethought to invent one's own private programming language, said job-for-life is doubly secure. But the associated disadvantage is quadrupled.

I was amused by the relevance to Conway and Kochen's Free Will Theorem when I came across Bramhall's reply to Hobbes:

"Will he ascribe liberty to inanimate creatures also, which have neither reason, nor spontaneity, nor so much as sensitive appetite?" -- Bishop Bramhall of Derry

If you've been re-formatting your SD flash card(s) for linux and don't know what an "erase boundary" is, you need to read Aligning filesystems to an SSD's erase block size and How to damage a FLASH storage device.

Let me summarise:

1. Don't repartition or reformat your SD card if you can help it.
2. Do this for ext3 (omit -j for ext2):

   fdisk -H 224 -S 56 /dev/sdb
   [partition as normal]
   mke2fs -j -E stripe-width=32,resize=500G /dev/sdb1
   [etc.]
                   

   ...or this for ext4:

   fdisk -H 224 -S 56 /dev/sdb
   [partition as normal]
   mke2fs -t ext4 -E stripe-width=32,resize=500G /dev/sdb1
   [etc.]
                   

   ...assuming your flash drive is mounted on /dev/sdb, of course.
3. If you have to, and you want FAT16 or 32, I can't help you but you're going to hurt the lifetime of your disk unless you act on the information in those pages; comment on them with the recipie(s) you've found to save others the time.

For my OLPC/XO-1 laptop these days, I'm using the interesting Sugar-on-a-Stick distribution, which is based on Fedora Linux. I want my XO to boot fast, not because I boot it a lot, but because it boots really, really slowly right now. So I installed bootchart, changed olpc.fth to use it, rebooted, and voila, got this chart:

.

Then I edited /etc/bootchartd.conf to use process accounting, and got this chart:

.

I booted without my SD card (which isn't the boot device - I boot off the NAND) inserted, and got this chart:

.

So there's some work to do to fix some SD card badness... Meanwhile, looks like I should get more familiar with rc.sysinit.

The data for the above charts are all available for download.

Could-be-interesting (though it's C# only; would love to see how IronPython scripting code would look) free / open-source trading system (purportedly): http://opentrader.org. I wonder if there are any others like it?

I have something like this in a "prepare-new-XO" script:

if ! `grep -q compcache /etc/rc.local` ; then
    set +e
    target_dir=/home/olpc/src
    this_kernel_rpm=kernel-devel-`uname -r`.i586.rpm

    mkdir $targetdir
    cd $targetdir

    /bin/rm -f ${this_kernel_rpm}*

    wget
    http://dev.laptop.org/~dilinger/{stable,testing}/$this_kernel_rpm
    wget http://compcache.googlecode.com/files/compcache-0.4.tar.gz

    rpm -ivh $this_kernel_rpm
    cp -af /boot/* /versions/boot/current/boot/

    tar xzf compcache-0.4.tar.gz
    cd compcache-0.4
    make
    ./use_compcache.sh
    echo "(cd /home/olpc/src/compcache-0.4 ; ./use_compcache.sh)" >>
    /etc/rc.local

    set -e
fi

Add a clock to your OLPC XO (build 767 - the current stable and G1G1 2008 version):

1. Launch Terminal and type "su -" (without the quotes)
2. Type EXACTLY, ALL ON ONE LINE (mind your spaces and minuses and capitals!)

   wget -O - http://www.xades.com/proj/clock_device_767.tar.gz | tar xvzf - -C /usr/share/sugar/shell
               

3. Restart Sugar by pressing Ctrl-Alt-Backspace (you will lose all your open programs and data, so be sure to save anything you want first)

Download lsprofcalltree from pypy; then do something like (thanks to Tomeu):

12:39 < tomeu> import os
12:39 < tomeu> import cProfile
12:39 < tomeu> import lsprofcalltree
12:39 < tomeu> profiler = cProfile.Profile()
12:39 < tomeu> profiler.enable()
12:39 < tomeu> ######################
12:39 < tomeu> profiler.disable()
12:39 < tomeu> k = lsprofcalltree.KCacheGrind(profiler)
12:39 < tomeu> data = open(os.path.expanduser('/tmp/import.kgrind'), 'w+')

...and throw that into kcachegrind.

The freedesktop.org Python DBus API documentation needs to be higher on Google searches for "python dbus".

I don't understand why irssi - a great program - doesn't come shipped with settings that cause joins/leaves to show up in a separate window. Evidently most of IRC is about the schadenfreude of seeing kickbanned people trying to bounce back into a channel or something.

It took me about five seconds to get annoyed by getting real conversations interrupted by messages - with extra color and/or highlighted - about whose computer had decided to go to sleep, or who had locked their screen. Is there so little real information in most channels that people like seeing this stuff?

To get messages that aren't part of the conversation out into one - the first - irssi window, just paste these commands into your irssi session:

            /set window_default_level -ALL MSGS PUBLICS NOTICES CTCPS ACTIONS KICKS MODES TOPICS NICKS CLIENTCRAP
            /foreach window /window level -ALL MSGS PUBLICS NOTICES CTCPS ACTIONS KICKS MODES TOPICS NICKS CLIENTCRAP
            /set window_check_level_first ON
            /window goto 1
            /window level ALL -MSGS
            /save
          

Job done. Read more hints in the Irssi documentation.

I'm trying compcache on my XO. To get it installed on my XO, from the console of my XO running the latest joyride:

sudo /sbin/init 3

#get joyride kernel-devel rpm
wget http://dev.laptop.org/~dilinger/stable/kernel-devel-2.6.22-20080408.1.olpc.de2a86ff3b60edc.i586.rpm
sudo rpm -ivh kernel-devel-2.6.22-20080408.1.olpc.de2a86ff3b60edc.i586.rpm
sudo cp -af /boot/* /versions/boot/current/boot/

#get compcache
wget http://compcache.googlecode.com/files/compcache-0.3.tar.gz
tar xzf compcache-0.3.tar.gz
cd compcache-0.3
make

#use it
sudo ./use_compcache.sh
          

I got a warning about udevadm being missing, and indeed it is/was and I have no idea whence to get it, but it's just used to wait until the right compressed ram device shows up, and that wasn't necessary in my case (presumably running use_compcache.sh twice would be good enough), so I'm not worrying about it. The results...this is before, in runlevel 5, with 2 Terminals, 1 Browse, 1 Record, and 1 EToys activity running:

-bash-3.2# free
             total       used       free     shared    buffers     cached
Mem:        237840     234036       3804          0        260      35268
-/+ buffers/cache:     198508      39332
Swap:            0          0          0
-bash-3.2# time yum search memory-pig

[1]+  Stopped                 yum search memory-pig

real    29m23.437s
user    0m0.000s
sys     0m0.180s
-bash-3.2# kill -9 %1
          

...and as you can see I had to kill yum after 29 minutes with end in sight, OOM killer having killed two of my python processes (one of which was Browse, and the other unknown...) and the XO becoming very unresponsive (well, I was running screen, three shells, and emacs (empty buffer) as well as vmstat 1 500 to show the memory usage, and the sys cpu percentage went to 95% for a long time).

This is after init 3; use_compcache.sh ; init 5, and launching all the same apps as before:

-bash-3.2# free
             total       used       free     shared    buffers     cached
Mem:        237840     232636       5204          0       1372      43776
-/+ buffers/cache:     187488      50352
Swap:        59456      13068      46388
-bash-3.2# time yum search memory-pig
olpc_development          100% |=========================| 2.4 kB    00:00
No Matches found

real    1m52.392s
user    0m16.250s
sys     0m27.430s

          

...so hey, I'll leave it installed for now :). How does it feel? Hmm...it's too early to tell.

I now have two patches accepted into the Sugar! They're about the smallest patches you can get - the first was one line, a removal :) - but it's nice to have something in the tree. There are a few still outstanding though.

Every time I try to do something in a Lisp requiring not rewriting huge libraries-worth of code I want to boot to the head the smartasses who rant about Lisp being a kickass language and fail to mention that packaging sucks and nobody seems to care that it takes days to figure out how to install each package because the maintainer got bored with other people using their code about 30 seconds after sticking it on CLiki. I imagine Esperanto advocates take the same type of (sadistic) pleasure in this type of situation: "Esperanto is so cool, everybody who's good would use it", I imagine them saying. "Where's Esperanto's Shakespeare?", I'd retort. "Don't worry, you can learn English from this crack-addicted tramp on speed with ADD", they would answer, "and then you can read Shakespeare, and then you'll realize why you shouldn't worry about him and instead be spending time coming up with baroque sentences of interest to abstract Esperanto researchers or arguing about which paper is best to browse the other Esperanto sentences of interest to more than two people in the world".

So I wanted to play with ECL and SBCL, so tried (require 'cl-ppcre) in sbcl. Argh, stale FASLs. Imagine the acronym "ABI". Now imagine the word "compatibility". If you're a typical Lisp hacker you're already distracted and I'd have no chance of combining those words with "shouldn't be broken by a patch release of your compiler/intepreter in this day and age".

After finding a nice .sbclrc snippet to try a recompile if a stale FASL was found, (require 'cl-ppcre) worked! (require 'split-sequence) didn't, though, nor did a few other ones I tried at random (most from memory but one while staring at its CLiki page!).

So then I found Lisp with Batteries Included 2 and tried to get clbuild going. It rebuilt sbcl (!?!?) fine -- amazing. Why Lisp package management is so broken that there's mk-defsystem, asdf-install, and super Lisp hackers (and I mean it certainly seems like these guys really know their way around Lisp development enviroments) have to create a third (!), will never cease to amaze me. There have been rants about how Lisp people are too smart to have their attention distracted by helping other people by fixing this type of problem, and then the counter-rants saying "it's easy enough; we all did it, and if one can't, then one isn't worthy!". Just recently I came across Steele or somebody like that mentioning how "mainstream" languages (by which I mean languages that are not such a pain to get to work with) were actually much more varied not in their language features (or other things a language designer would typically worry about) but varied in their community ("culture?") and their library support. He got it right. And Lisp makes me cry each time I think of how it measures up in those two areas.

All this wouldn't even have been attempted if the implementation language wasn't so cool. It's like having a beautiful mechanical toy whose mechanisms are so interesting to even just glimpse from time to time that you're happy to put up with having to clean up the crap that seems to spontaneously accrete every time you come back to it before you can even see the beauty.

I'd mark up this text with proper links to the referenced articles/tools but it's five AM so I'll leave it for another day when I don't have to recompile sbcl. I'm still amazed that it worked the first time; see! The code is great but the inconsistency of the tools...argh.

LIFO - my last two patches (to support-scripts [1] [2]) just got added. They're small patches, but useful for scripting and good of the maintainer (Phil Bordelon) to merge them so quickly.

My OLPC patches are still being digested; I think update.1 work is keeping people focused elsewhere (given I recently submitted a one-line patch to bug #4646, I don't think it's (just) because my patches are rubbish). So in order to make it a bit easier for me to enqueue something on my XO for offline reading, I combined:

• My patched copy-to-journal
• Simple bash script to scp files and call copy-to-journal
• nautilus-actions integration of said script. Once configured, it's pretty easy to use the nautilus-actions-config control panel to configure nautilus-actions to run this script via a right-click menu (unfortunately there is no shortcut/accelerator support yet) in nautilus.
• I also used mozex to automate putting a link in the journal with this script. Nice.

Self-organizing maps look cool, but not (mainly) literally: using them for categorizing price movement patterns is the subject of some research, but not really very much. 2-5- and 10-20- (trading-) day-long patterns of "special situation" stocks could be interesting.

I've gone out on a limb (because it's kind of none of my business) with some patches to Sugar that make nano/emacs usable from a sugar terminal via the standard keybindings (like, C-s and C-o actually work).

I really need to learn more about continuations, like Self-application as the fixpoint of call/cc

My ebook mode patches appear to have attracted no attention :(. I hope it's just because there is a big XO deployment and people are busy. Maybe I could make the initial focus on the Browse and Read applications be a lot more sensible (and ebook-friendly) in the meantime.

I submitted some patches to better support ebook-mode on the XO. Any python activity will fullscreen itself and hide its cursor, which is pretty much just what you want when you hit any buttons on the screen or move the mouse (because you're in ebook-mode).

Recently there was some publicity about XP on the XO. I think some of the reaction to the reaction (!) was regrettable, and I said so:

I think you guys just got it wrong, here:

1) What is OLPC getting? I'm not sure what OLPC will be getting out of
   helping Microsoft: you say that you're not dedicating resources to
   it, but I bet if 5 random developers showed up and asked for the
   same amount of time from you (meetings + technical writer) they
   wouldn't get the same reception.

2) What are you losing? Your time (meetings) and others' support for
   the project (bad PR), but most importantly, the ability of the
   laptops in the field to be used for OLPC core principles
   (trivially, the 5th, but given the state of mesh networking now, I
   can't imagine trying to get things working with Windows will speed
   up achieving saturation and connection). I agree with Bruce about
   the 'paradox of choice' that you seem to ignore (or straw man the
   other side - 'To claim we should prohibit XO customers from running
   XP') is that the choice that removes your ability to choose is no
   choice. I know you said they must be able to switch back to a free
   alternative (Linux), but if you really believe that once officials
   have the choice to distribute Windows the sales (bribery?) pressure
   won't kick in and leave kids with Windows laptops that their
   parents can sell or DRM'ed textbooks they need to have, I submit
   Intel's latest antics as the obvious riposte.  3) Your point about
   the Microsoft people you're dealing with being 'good people', and
   want to 'play along', goes nowhere concrete so I assume it's meant
   to convince you/us that this is all a warm, fuzzy,
   help-the-kids-first project. If so, doesn't that seem a bit naive?
   If not, please help me understand where you were taking that point
   (I assume it wasn't 'MS supports OLPC's five core principles; look,
   these nice people are just the beginning').

I only hope this is just a friendly, developer-to-developer set of
interactions that got too much press. Given Microsoft have had 40
people working on this for a year, I doubt it, and I think you've gone
too far. Let Microsoft employees join the devel list and exchange
their ideas in the open, at least, on the same playing field as
everybody else.

Martin

PS - you also lost me when you said OLPC's 'commitment to open source
     and free software isn't religious, but pragmatic' - if one of
     your five goals can be cast aside so easily for 'pragmatic'
     reasons, I think you're going to lose a bit of people's trust and
     goodwill. I would have expected one of your core goals would be
     adhered to except in extraordinary circumstances (c.f. the
     Marvell firmware). These are hardly extraordinary circumstances,
     except in the naivite that what Microsoft has done with other OS
     competitors won't be done to OLPC.

I figured out how to control the XO laptop's backlight from another's blog and thought I'd re-highlight and expand on it here:

The backlight levels can be modified by writing to /sys/class/backlight/dcon-bl/brightness. The backlight seems to have 15 levels of color-mode brightness (1-15) and one level indicating black and white mode (0).

Here're examples of how to modify the backlight levels from the Terminal:

            sudo bash -c "echo 0 > /sys/class/backlight/dcon-bl/brightness"
            sudo bash -c "echo 15 > /sys/class/backlight/dcon-bl/brightness"
          

I've got an XO Laptop (from the OLPC project) and it's pretty cool. I've upgraded the software (joyride-1502 works fine for me) and figured out how to get the battery capacity into a gnu screen hardstatus line with this tiny script & screenrc change:

---------- /home/olpc/bin/olpc_screen_status.sh
#!/bin/bash
# Author: Martin Dengler <martin@martindengler.com>
# idea from http://www.mail-archive.com/screen-users@gnu.org/msg00322.html
# based on battery info from olpc-logbat
#
B_INFO=/sys/class/power_supply/olpc-battery

while true
do
    CAP=`cat $B_INFO/capacity`
    echo b:$CAP%
    sleep 60
done
----------
          

....screenrc change:

---------- /home/olpc/bin/.screenrc
# run command forever and assign most recent output to string escape %1`
backtick 1 0 0 /home/olpc/bin/olpc_screen_status.sh

# the ...%1`... part at the end is the important part
hardstatus alwayslastline "%{.bW}%-w%{.rW}%n %t%{-}%+w %=%{..G} %H
%{..Y} %m/%d %C%a %1` "
----------
          

Some new famous quotes (from norvig.com):

The quality of a person's life is in direct proportion to their commitment to excellence, regardless of their field of endeavor.
    - Vince Lombardi

It is the mark of an educated mind to be able to entertain a thought without accepting it.
    - Aristotle

Believe, if thou wilt, that mountains change their place, but believe not that man changes his nature.
    - Mohammed

Recently had a conversation at a meetup about what RSS feeds we read. I heard about a few new ones; here are mine:

• YC Hacker News
• xkcd.com
• Lisp/Scheme/Emacs Yahoo! pipe from Bill Clementson
• Quantitative Trading
• The Register
• Unofficial Paul Graham
• Eric Lippert's blog

Peter Siebel recommends people link to this Common Lisp tutorial because there isn't anything better! Seems like the least we can do.

I'm interested in buying a laptop or two. Both have to have:

• Free/LinuxBIOS / OpenBIOS support
• good linux support (Fedora)

I might want two, one as a text-only hacking/emailing console-about-the-house and another as a desktop replacement.

I'll have to update my projects page as I do the research.

I've updated my Fedora repos page for Fedora 8.

If you're thinking of upgrading/installing Fedora 8 without having to burn any CDs/DVDs and have an extra hard drive partition (usb disk? home partition you won't be blowing away?), just drop the rescue disk's initrd.img and vmliunz into your existing /boot directory and add a new entry to your /boot/grub/grub.conf pointing to them. Store the DVD iso on the partition that will survive the upgrade and reboot. Make sure to select that new grub entry when you reboot, and you should be off and installing.

I've been messing around with my .emacs file to make it useful on a number of machines. I figured I'd mention it in case I need to copy it somewhere without scp or some search engine can use it to help someone someday.

It's got a few standard customizations and cute stuff grabbed from around the web:

• global auto-revert and font-lock mode
• UI stripdown (a bit)
• slime
• iswitchb
• with-library macro
• tramp mode

A long time ago, Kent M. Pitman mentioned a list of langague-classes (and examples thereof) that a programmer should get experience with to make them better programmers. I checked off those that I know and was pleasantly surprised:

• A block-structured language, such as Algol or Pascal or Scheme.
• A line-oriented language, such as Fortran or BASIC.
• A rule-based logic language, such as Prolog or EMYCIN.
• An Englishy language such as HyperTalk or Cobol.
• A "stack" language such as PostScript.
• A "line noise" language (heavy emphasis on one-letter operators) like APL or Teco.
• A dynamic object-oriented language, such as Common Lisp or Smalltalk.
• A strongly-typed, statically analyzed language such as ML or Haskell.
• A dynamically-typed functional programming language, such asScheme.
• A string or macro processing language, such as Perl or m4 or TeX orTeco.
• A database access language, such as SQL.
• An abstract high-level, assembly language, such as Java.
• A concrete high-level, assembly language, such as C.
• A low-level, traditional assembly language.
• A scripting language, such as Javascript.
• An interface-definition language such as Visual Basic, HyperTalk, or Javascript.
• A document-structuring language such as XSL or TeX.
• A language with a modern error system, such as Common Lisp or Dylan.
• A reflective/introspective language such as Common Lisp or Java.
• A symbolic programming language, such as Common Lisp or Scheme.
• A security-oriented language, such as Java or MOO.
• A language where both programs and data are fully persistent, such as MOO or HyperTalk.

I've cleaned up this first page. 2003-2006s' page is still available.

Great Lisp comic. If they showed more CS majors that they'd overcome a few of Lisp's disadvantages

Backups are for wimps: Real men, and all that.

Just a small script to control my TV (LG 37Z55). Really simple example of pyserial usage, but not supposed to be anything like a model bit of code (it's not interesting enough, first off). Will probably work with other LG models.

The closest I've come to finding a reference to the rabbit problem: Programming puzzle.