Skip to content

monkeyYaml part 2, or unexpected longevity

09-May-24

Ten years ago, I hacked out a limited parser in Python for YAML, which I named monkeyYaml. I even blogged about it.

It turns out that this code is still in use. Not only as a backup, but as the *only* parser for front matter in the test262 repository.

Initially, I used import yaml to parse the YAML front matter. When this change broke on some of Google’s vast Chromium test farm, on machines with Python 2 where yaml was not available, I wrote monkeyYaml as a backup parser.

Shortly after that, though, user jugglinmike (or, as I prefer to think of him, MegaMan) pointed out that using two different parsers was risky. A new test author might use a YAML feature not supported by monkeyYaml, and if they tested on a modern installation of Python 2 or Python 3, it would appear to work… only breaking when it eventually reached the Chromium test farm.

So, to ensure that we stay within the YAML subset supported by monkeyYaml, MegaMan made this change to ensure that monkeyYaml would be the only parser, not the fallback parser.

Don’t Know Much Philosophy…

08-May-24

I really like my idea of what philosophy is supposed to be. Unfortunately this doesn’t seem to align with what philosophy is to other people. Notably, I took two philosophy courses in undergrad and earned a D in the first and a D+ in the second.

But now I’m taking a sabbatical, and I am going to use my sabbatical to read, write, think, and discuss philosophy, and maybe figure out what feels discordant between my idea of philosophy vs. the general meaning.

To kick this off, I decided to read Philosophy: A Very Short Introduction, to get an overview of the topic.

But my friend recently reminded me of the existence of Gell-Mann Amnesia, so before I read that book, I first read Chemistry: A Very Short Introduction. That book, by Peter Atkins, turned out to be very good — I highly recommend it, if anyone wants a very short introduction to one of my favourite things.

So now, I am partway through the Philosophy book, and enjoying it quite a bit; enough to expand what I think my idea of philosophy is, and how it separates from modern academic philosophy.

Recalled To Life

08-May-24

Hello again! It’s been almost ten years since I last posted here.

Back in 2014 I was working on a chemistry app for pre-medical students; my idea was to make it easier to understand and remember enough organic chemistry to do well in the required classes and on the MCAT exam.

I realized that it would be too much work for one person to do competently. I couldn’t even do all the coding, design, backend administration etc – let alone the marketing, user support, UI design, content generation, and all the other things necessary to make the app successful. I would need a team.

But I didn’t know how to assemble a team, how to inspire a group of people to all work on a common goal and keep going in the same direction. I didn’t know how to manage people.

I didn’t even know if I would be any good at it, since I’d never tried it before. And I didn’t know whether I would like it.

In 2015, I got an unexpected opportunity to work for a big tech company. I jumped at the chance, and joined as a senior developer. I told my manager that I was interested in trying management, and after a few years I became a manager; then in 2021 we split off my team and I had the opportunity (well, the necessity) to hire a lot of people in a short time.

It turns out that I really like managing developers.

Like Poor Old Ollie

17-Oct-14

I was thinking some more about the new game “Shadow of Mordor” (read the review by Carolyn Petit here). I can’t really add anything to her review — she makes it sound like the plot of God of War was transplanted into an Assassin’s Creed game set among some of Tolkien’s ideas — not truly in the Tolkien universe.

I wanted to criticize Christopher Tolkien and the literary estate for allowing this to happen, but then I read this article about the Tolkien Estate’s battles with Warner over digital content, and I saw that the Estate had accepted a percentage of the net film proceeds — which is incredible to me. Everybody knows that films don’t make money on net, and if you want to get paid you need a piece of the gross. This is not news.

So I started thinking about CT’s age; he sounds pretty sharp in the interview, but the guy was born in 1924, and he licensed the films in something like 2004, so he was 80 then and he’s 90 now. So I’m now I’m just feeling down about the whole thing.

I bet when CT dies, Warner will execute their option to dig up the remains of JRR Tolkien, hang him, and then burn him, like the royalists did with Oliver Cromwell.

Monkey-YAML (or: Reinventing the Wheel)

15-Oct-14

In theory, we’re never supposed to reinvent the wheel. In theory, we’re supposed to fix things correctly.

Well, as Yogi Berra allegedly said, the difference between theory and practice is that, in theory, there is no difference…

Some time ago I got involved in the Test262 project (you should too!) and, one thing leading to another, I signed myself up to make the Chromium project’s test runners work with a new version of Test262’s test cases.

The test cases had switched from a custom metadata format to YAML, a well-designed and well-supported format. Well-supported, that is, except in every python installation on each of the thousands of machines that the Chromium project uses to run its distributed testing. Because py-YAML is not distributed with python, it’s not safe to use for the Chromium test runner.

So I wrote a fallback, in case py-YAML is not installed. I wrote my own, very rough, parser that parses a subset of YAML — the subset Test262 uses — and returns the same results. The test suite looks a lot like this:

def test_es5id(self):
    y = "es5id: 15.2.3.6-4-102"
    self.assertEqual(monkeyYaml.load(y), yaml.load(y))

Set up a string, and then ensure that my parser (monkeyYaml) returns the same result as the real YAML parser. This way I can ensure that my parser functions as a drop-in replacement.

I’m pleased to say that this hideous hack was recently incorporated into the official test262 repository. Because sometimes a hideous hack that reinvents the wheel is exactly the right thing…

Installing node.js 0.10 on CentOS5

09-May-14

CentOS5+Node I have an old stable internal server that’s running CentOS 5.10 which I want to use a staging/development server for modern node.js applications.

The problem is, current prebuilt versions of node.js can’t be installed on CentOS5, since CentOS5 has GNU libc 2.5 and the prebuilt node.js RPMs are only for Fedora/RedHat/CentOS 6, and are linked against libc 2.7.

Fortunately you can still build from source, as follows:

Install Python 2.7 on CentOS5

Node requires Python 2.6 or 2.7 to build; CentOS 5 comes with Python 2.4, and we need to keep that because yum depends on it.
So we first install Python 2.7 as an alternate version of Python:

$ cd ~/code
$ wget https://www.python.org/ftp/python/2.7.6/Python-2.7.6.tgz
$ tar xzf Python-2.7.6.tar.gz
$ cd Python-2.7.6
$ ./configure; make -j
$ sudo make altinstall

It’s very important to do make altinstall and not a regular install, because yum depends on Python 2.4. make altinstall will install Python 2.7 as /usr/local/bin/python2.7

Check out latest stable node source

$ cd ~/github
$ git clone git://github.com/joyent/node.git
$ cd node
$ git checkout 0.10.28
$ PYTHON=python2.7 $PYTHON configure
$ make
$ make test

I had a few failing tests. The very first test fails, and takes many seconds to timeout. This is test-abort-fatal-error.js, which seems to be testing the system’s ability to apply resource limits via ulimit. Another test that fails, near the end, is tls-test-server-verify. This sounds ominous, but since I am not planning to use this server in production, I decided to continue.

$ sudo make install

Test node and npm

$ node --version
v0.10.28
$ node
> console.log('Hello, world');
Hello, world
undefined
> ^D

Then create a directory to test npm

$ mkdir test && cd test
$ npm i --save mocha
$ PYTHON=python2.7 npm i --save inchi

Note the explicit setting of the PYTHON environment variable. When installing an npm module that requires compilation (that executes node-gyp), npm will return an error if it runs with the default system python (2.4), since node-gyp requires at least python 2.7. If you install a lot of compiled packages, or it’s painful to remember to set the environment variable PYTHON every time, you can add export PYTHON=python2.7 to your ~/.bash_profile. It doesn’t seem to bother yum.

Inchi, by the way, is an international chemical identifier; specifically, the npm inchi package is javascript interface to the InChI Trust’s C library for converting between structures and InChI codes. Here I’m using it as an example of an npm module that requires compilation and linking to work.

I also noticed that when node-gyp was building

Known Limitations

There are several limitations to this approach.

  1. Test failures making this node 0.10 unsuitable for production

    The ulimit, tls, and flooding tests failed. I would not use this node on a production server that is exposed to DOS attacks.

  2. No support for nvm

    On my production server I use nvm to manage the version of node I’m using. Since nvm downloads prebuilt binaries which link against glibc 2.7, this doesn’t work on CentOS 5

  3. Must rebuild node to get bugfixes / new versions

    Just like in the 1990s, in order to get the next stable version of node, you need to pull the sources and rebuild. This is much easier now, and can even be automated, but it’s a substantial effort that only makes sense if you have no better alternative.

If I can get the tests to build cleanly, I’d be willing to package this up as an RPM for anyone who’s interested.

EDIT: Here’s another approach, written for node 0.8, but will probably work with 0.10. Note that they use EPEL to find and download an RPM for Python 2.6, and a symbolic link from ~/bin/python to /usr/local/bin/python2.6.

Chemical Structure QR

24-Apr-14

I’m holding a copy of Nature Chemistry in my hands and I see an interesting-looking molecule. Here, you take a look: 20140424-070829.jpg

Cute, eh? I can think of a number of ways to use this molecule in undergrad chemical education:

  • Identify the functional groups

  • Calculate the degree of unsaturation

  • Name R/S stereochemistry

More difficult problems could be

  • Name the compound

  • Predict the products under given reaction conditions

  • Give reaction conditions that would produce a specific product.

Functionalizing the phenyl ring would be especially good to ask about, since it would reach into resonance stabilization through the alcohol function, possible interference from the cyclopentene double bond.

If I was feeling really mean I’d ask for a retrosynthetic analysis… using only terpenes.

One problem, though. What is this molecule?

It’s easy enough to draw — I just timed myself at 11s. On paper, that is. But naming is a bit annoying, especially since I haven’t really been using the ‘organic nomenclature’ portion of my brain for the last fifteen years. It’s a 1-ol, phenyl, 2-(cyclopent-2-enyl) … propane? Screw this, I’ll look it up in PubChem!

Ten minutes of fighting with the pubchem structure drawing engine reveals that the compound was deposited as (S)-[(1R)-1-methylcyclopent-2-en-1-yl]-phenylmethanol … deposited by the good folks at Nature Chemistry, in fact. Ah yes. Actually PubChem has a convenient link right to the article, from which I can download the Supplementary information PDF, which contains a slightly different name– ((S)-((R)-1-Methylcyclopent-2-enyl)(phenyl)methanol) and a slightly different diagram (with an explicit Me at the 1-carbon of cyclopentene). And if I drive all the way through the full text of the online article (or Google the InChIKey, GZJFLEKXATUYSK-OLZOCXBDSA-N), I can find that this compound actually has its own home page on Nature Chemistry, with a link to PubChem and downloadable representations of the molecule in ChemDraw and mol formats.

I’m not saying this to single out Nature Chemistry, by the way. This is great stuff, and much better than the way it used to be — I am young enough that although I was trained in how to use Chem Abstracts, I was fortunate enough to never need this knowledge.

But what if the article had been printed with a tiny QR code in it, like this: qrcode.21841394 This encodes the InChIKey, from which you can easily do a PubChem or Google search. A chemical structure QR code doesn’t just have to be the InChiKey; a journal could include a link back to the figure or compound’s home page.

Try it out yourself – you scan scan QR codes from your screen natively on an Android phone; with iOS you need to install an app (I use the ‘free’ ad-supported version of Qrafter)

Imagine if you could easily include this QR in every chemical structure drawn in ChemDraw. Imagine if, while going through a stack of reprints or preprints or articles to review, you could easily get to an online structure database and search the literature around a particular compound.

Workcation

01-Mar-14

I’m starting a week-long work block today — that is, I’m planning to work from 5AM to 5PM straight every day, like a “normal person”. Danielle has graciously consented to spend one week of her vacation doing everything else that needs to get done.

Since last week we had a “stay-cation” where we went out to all the Edmonton sights and attractions and ate out at fancy restaurants, I have dubbed this my “work-cation” – which is not pronounced “work cat ion”, for the chemists out there. That would just be anion– err, annoying.

Here’s my day 1 setup:

I moved the mac into the guest room for my workcation.  That's ~1L of coffee on the right there, in my homemade skull mug and Contigo travel mug.

]1 I moved the mac into the guest room for my workcation. That’s ~1L of coffee on the right there, in my homemade skull mug and Contigo travel mug.

And here are the materials I brought along:

Notes, books, flashcards, papers for my 2014 March work-cation.

]2 Notes, books, flashcards, papers for my 2014 March work-cation.

Books I recommend to everyone

07-Feb-14

My friend Evan asked about design books and I spouted off a few – including The Systems Bible, which I now feel guilty about, because it’s not strictly a design book.

I included it because it talks about how system design fails, and it’s humorous and I think teaches some larger truths. It was originally published as “the general theory of systemantics” — a portmanteau of system and semantics — but also a pun, since the first law is that “a system displays antics”.

Here are some more book recommendations, broadly disorganized.

In politics and philosophy:

  • Karl Popper, The open society and its enemies (2 volumes)
  • The Road to Serfdom, FA Hayek
  • In general systems design, work, and sociology:

  • The Systems Bible
  • The Peter Principle
  • In small business

  • Growing a Business
  • Under The Radar
  • In schooling

  • How Children Fail
  • How Children Learn
  • Instead of Education
  • Deschooling Society by Ivan Illich
  • In government

  • unchecked and unbalanced
  • In drama and human behavior

  • Impro by Keith Jonstone
  • In parenting

  • Playful Parenting
  • Raising your Spirited Child
  • In homeschooling

  • Teach Your Own
  • The Well-Trained Mind
  • In literary analysis (and a little sociology)

  • The Outsider by Colin Wilson
  • In design

  • A Pattern Language
  • The Non-Designers Design Book
  • The Design of Everyday Things
  • In self-improvement

  • Getting Things Done
  • The Procrastination Puzzle by Tim Pychyl
  • Good Mood by Julian Simon – an unusual approach to CBT
  • How To Win Friends etc. – Dale Carnegie, still the classic
  • the happiness hypothesis – Jon Haidt
  • In Military History

  • A History of Warfare
  • The a face of Battle
  • Carnage and Culture
  • The rise and fall of the Third Reich
  • in General history

  • The Story Of The World – 4 volumes ; aimed at children aged 7-11
  • In Cognitive Psych

  • Metaphors We Live By
  • applied psych/relationships

  • The five Love Languages
  • Men are From Mars – included because a dear friend recommended it to me after his divorce and said “if only I had read this before, I might have been able to save my marriage”
  • the Good Marriage
  • Divorce & new Beginnings
  • Passing options to node on the shebang (#!) line

    01-Feb-14

    I was chatting with someone on #node.js who wanted his script to pass a command-line option to node, so that his script was run in a particular node environment. The problem is that under linux you get to pass exactly one argument on the shebang (#!) line. If you use #!/usr/bin/env node, you’ve already used your one argument. When I suggested he use the “-x” hack, we discovered that node didn’t have this hack. So I made a pull request complete with a TL;DR justification for why -x is necessary.

    Turns out there’s a tidier hack that doesn’t require any changes to node, which relies on the interaction between bash and node.  Here’s an example, lifted from pm2 and lightly modified for clarity:

    #!/bin/sh 
    ":" //# comment; exec /usr/bin/env node --noharmony "$0" "$@"
    
    console.log('javascript');
    

    Here’s how it works:

    1. The #!/bin/sh causes the script to be identified as a shell script, and passed to /bin/sh for execution. /bin/sh reads and executes scripts one line at a time, and we’re taking advantage of that below.

    2. The second line, as interpreted by the shell, consists of two commands.

      2a. The first command is ":", which is the quoted version of the rarely-used bash command :, which means “expand arguments and no-op”. The only argument to : is //, which is a valid path. The following # is a bash comment, which is valid until the command separator ;.

      2b. The second command is exec /usr/bin/env node --noharmony "$0" "$@" which executes the node interpreter with the desired arguments and passes argument 0 (this script file) and the rest of the arguments to the bash script ("$@")

    3. The exec causes the bash process to be replaced by the node process, so bash does not attempt to process any further lines.

    Now we’re running under node, with the desired command line arguments set. Unlike bash, node wants to read and parse the whole file. So let’s see what node sees:

    1. The #!/bin/sh line is ignored due to a special one-off in node – when loading a module, the contents of the first line will be ignored from #! up to the first \n.
    2. The second line contains a string constant, the quoted string ":", followed by a Javascript comment introduced with //. Automatic semicolon insertion happens so the constant is interpreted as a string in a statement context. Then the comment is parsed, and everything up to the end of the line is ignored by node.

    This won’t lint clean. jslint and jshint both complain:

    $ jslint test
    test:2:1: Expected an assignment or function call and instead saw an expression.
    test:2:4: Expected ';' and instead saw 'console'.
    $ jshint test
    test: line 2, col 1, Missing semicolon.
    
    1 error
    

    But it works right now, as a hack-around for the Linux one-argument shebang problem.

    Note that there’s a spot in the line where you can insert a comment (as long as it doesn’t contain anything that bash interprets, notably ;). What to put there? I recommend a link to a web page (such as the one you’re reading now, https://sambal.org/?p=1014) that explains WTF this weird-looking line is all about. For example:

    #!/bin/sh 
    ":" //# https://sambal.org/?p=1014 ; exec /usr/bin/env node --noharmony "$0" "$@"
    
    console.log('javascript');
    

    Happy hacking!