Terse Systems

I moved all my note taking into Evernote, and I’m using it for everything – diaries, documents, clippings, etc. Much simpler and more powerful than using three different systems.

I had to write some Ruby scripts to get everything imported: they are available at http://github.com/wsargent/evernote-import. Imports from plain text files and from Memiary XML format.

I read The New Executive Brain yesterday, by Elkhonon Goldberg. It’s the closest thing to a professional viewpoint of the brain that you’ll find. It can only barely be considered a popular science book. It’s just too dense. Goldberg is fiercely intelligent, and he doesn’t stint on the vocabulary or the terminology. There were times I’d read through a chapter and have to put the book down because I couldn’t absorb any more information.

It’s not a book about how people think. In fact, it’s explicitly not about people at all; it’s about how the brain works. It’s a book that explicitly talks about the brain like a car engine; here’s the amygdala, there’s the frontal lobes, and here’s the hippocampus. Put them together and you have a functional human being. Destroy a piece and you end up with a broken toy of a human being that can only ever go around in circles.

Destroy one part of the brain, and there are dozen different ramifications, from Attention Deficit Disorder, to Tourette’s, to Obsessive Compulsive Disorder. This is an excellent way to understand the brain as an organ, but it’s hard to maintain the idea that these disorders are “brain” and not “personality.” But to Goldberg, personality is an outcome of various interactions of the brain. He even complains that when describing flaws in the brain, people absorb it and then ask why the patient’s personality has changed – to Goldberg, he’s just explained why.

Much of the book is concerned with the frontal lobes, the “executive” part of the brain that is responsible for making conscious decisions. It turns out that damage in almost any part of the brain can cause malfunctions in the frontal lobes – this is because the frontal lobes receive input from almost every part of the brain to make decisions. Change the input, and you get output that doesn’t make sense. Exactly how the frontal lobes do this is unknown, but we know that both hemispheres of the brain are involved, and interact differently.

Everyone has heard the old story about being “left-brained” or “right-brained” where the left hemisphere is the logical side and the right hemisphere is the intuitive side. The true difference is more subtle than that. The left hemisphere has many neurons that connect to their close neighbors. It can pick up new behavior very quickly. The right hemisphere has many neurons that connect to far off neurons. It picks up new behavior more slowly than the left hemisphere. If the left hemisphere is a tightly woven net, then the right hemisphere is a much larger, courser net, with many scattershot branches.

They both function the same way in recognizing patterns, but when given a new task to learn, the right hemisphere is most active to begin with. New activity is captured and seen using the larger net. Then, as the task is learned and understood, activity migrates from the right hemisphere to the left. What was previously understood in an unformed, loose way, is seen and codified on the left hemisphere, which can recognize the pattern immediately the next time it sees it and deal with it appropriately.

This approach gives the brain the best of both worlds; an immediate loose understanding of a situation, and an efficient grasp of a well known situation. When modelling neural networks on computational platforms, it turns out that this combination gives very effective performance. It also explains how damage in the right hemisphere can be absorbed without obvious incident, while damage in the left can be so crippling; the neural network on the right hemisphere can route around the damage and heal without losing already acquired knowledge.

So. Good book. Recommended. Only read it if you really want to know.

Okay, so. I’m going to assume that everyone has now seen Avatar, and hence I’ll assume you’re all up for massive spoilers, etc.

The movie we all just watched was not the movie we thought we watched. And this is not just some George “hahaha, I destroyed the hopes and dreams of a generation” Lucas screwing with the fanbase… this is lurking beneath the surface of Avatar from the beginning. The existence of the Na’vi.

The Na’vi have no common morphology with the rest of the planet. Sig even comments on it in a Youtube article. Why do they exist? How is it that they speak a recognizable language, and have genes close enough to human that it’s possible to MIX IN HUMAN DNA with the Na’vi? How is it that the Na’vi have built in neural interfacing equipment that can instantly domesticate the larger animals and even predators? Wouldn’t evolution make such a thing impossible?

The answer is that the Na’vi aren’t a natural race. Eywa made them. They’re close enough to human that the humans can communicate with them and think they look cute and cuddly (Ewya may have been slightly confused here), and alien enough that they can survive in the local environment. If that wasn’t enough, Eywa provided with some elevated sudo privileges, so they could take advantage of the local fauna without Eywa being directly involved.

The how is easy. Eywa’s a worldmind capable of transferring human neural networks into Na’vi clones on the second try, and it’s entirely feasible to create a race and set it up with false memories. And something that’s smart enough to create room temperature superconductor in bulk (what? You think Eywa runs on plants alone, when there’s massively complex electromagnetic flux happening around the tree and the Na’vi just happening to be sitting on giant deposits of unobtainium?) will have no problem reading our electromagnetic communications. Eywa’s on Alpha Centauri; it’s been listening in since the first radio broadcasts.

The why is a bit harder to explain. Why would a worldmind play dumb?

Well, probably because it has a very good understanding of what happens to things that look like a threat to humanity. If we had any conception of what Eywa is, we’d be terrified, and we’d probably sterilize the entire planet before we even set foot on it. As it is, Eywa looks harmless. It looks beautiful. It’s not exactly friendly, but it’s the kind of environment that keeps humans focused on the trees instead of on the forest. Eywa can afford to watch and wait until it has to act.

It also makes a hell of a way to see human capabilities up close and personal. Eywa is well capable of doing a vacuum cleaner impression on every single EM communication on the planet. And when Eywa saw a human built Na’vi run out the container and disregard orders, it could lay bets that this was someone stupid and romantic enough to provide Eywa with more insight into the military command structure. The cute floaty things aren’t accidental. They are Eywa’s way of saying “hold up, this guy could be useful.”

And Eywa gains massively out of it. Not only does it have a neural imprint of a woman with a massive amount of scientific knowledge, it also got to see a run through of the military. It had to expose itself to a certain extent to get the humans to back off, but you can be certain that the humans will have Eywa bacteria in their digestive tracts when they reach Earth. I wouldn’t be surprised if Eywa had a very specific form of toxoplasmosis all ready to go if needed. (Indeed, one plot treatment specifically mentioned this possibility.) It doesn’t have space travel down, but as long as it stays quiet, it can build up and infiltrate the military before they get to the point they know what they’re dealing with.

And for those romantic souls thinking this was about Jake Sully… dude. THEY LOST. Jake and the Na’vi didn’t stand a chance against the military, they used up most of what they had, and THEN, as soon as it looked like Eywa might actually lose a valuable resource, it acted. It was perfectly happy to use the Na’vi as cannon fodder if it meant it didn’t have to show a card.

And I, for one, welcome our worldmind overlord. Life under Eywa would not be a bad thing for the vast majority of people, and Eywa would make better use of our technology and infrastructure than we could. Eywa keeps the reins loose for the most part, and doesn’t enforce behaviour as much as “convincingly persuade.” That may be the Na’vi, the charming public face presented by the unknowable, near-omniscient power that is Eywa, but even if the Na’vi are rolled up and tossed out like last week’s pizza once we are no longer a threat (which is unlikely – the Na’vi are too damn useful for offworld activity), then humanity, and Earth as a whole, can rest in peace knowing that something much smarter, tougher and biologically engineered to perfection took us out.

If you want to know what makes you happy, you have to be willing to think hard about what happiness is, and pay attention to what makes you happy.

“Happiness comes in small doses, folks. It’s a cigarette butt, or a chocolate chip cookie, or a five second orgasm. You come, you smoke the butt, you eat the cookie, you go to sleep, wake up and go back to fucking work the next morning, THAT’S IT! End of fucking list!” - Denis Leary

So after much time and experience, here’s the list of things that make me happy.

1) Direct sunlight.
2) 8 hours of sleep.
3) Movement outside.
4) Social interaction.
5) Regular meals.
6) Satisfying work.

The upshot from this list is that I’m not all that complicated. Also, what actually makes me happy may not be what I spend most of your time thinking about. I don’t think about sunlight all that much. But I can tell the difference between when I have it and when I don’t. Biking has a huge effect on my mood. Not eating has a huge effect on my mood. And I’m going to take a leap in logic and say that this list is globally applicable to all humans.

The complicated part of this is social interaction and satisfying work. But even the satisfying work is simpler than you might expect.

But hang on a sec. This list doesn’t just apply to humans. Look at dogs, for example.

Dogs need sunlight. Just ask one.
Dogs need sleep. They get cranky if they don’t.
Dogs need walkies.
Dogs need to meet other dogs and sniff each others butts. And humans.
Dogs need regular meals. And they’ll eat anything you give them.
Dogs need to do something. Pointers need to herd, bloodhounds need to sniff.

You break it down and you’ll conclude that human beings are social animals, and have the same needs as social animals. I used to think that the people who get up at 8 am, eat breakfast and then jog for an hour were obnoxiously happy people who were just naturally gifted. They’re not. They’re happy because doing those things will make you a happy human. Likewise, staying up until 3 am, not getting outside, getting crappy sleep and reading existentialist philosophy will make you pretty damn unhappy. It doesn’t matter what your brain thinks about the activities you’re doing – do these things and you’ll look and act like an unhappy person the next day.

Again, this goes back to how to get the most milk out out of cows or the most work out of programmers.

The Dog Hypothesis: Human beings need walkies.

I started a job at Grockit a couple of weeks ago. This has been my first chance to see what an agile shop looks like, and it’s an eye opener.

You show up at 9:15 am every morning. There’s a daily stand up meeting (truly stand up, everyone is around a circle), and then breakfast, which is served in house; people sit down and chat about stuff they’re doing, but it’s less formal than the standup is. (I don’t have a good handle on how retrospectives or user story planning work, but from what I can tell there’s an overall direction that is set during releases, and then the business owner participates in standups to ensures that everyone’s on the right page. Spot demos also appear to be frequent, although I don’t know how much of this is a conscious strategy.)

Following the standup, there’s a secondary huddle where engineers look at the list of user stories open, and pick off the stories that they want to tackle. Pairing is done by the tech lead rather than the individual engineers, and there’s an emphasis on switching pairs around. Tasks get written down on the whiteboard, photos with names are assigned to them for the day, and then it’s time to find a free workstation and move in.

Pair programming is the natural order of things at Grockit. The workstations (Mac Pros with 30 inch monitors) are all set up in exactly the same way (IntelliJ IDEA with Ruby Plugin, QuickSilver) and have two keyboards and mice attached. There are two engineers per desk. The first thing you do is bring up Pivotal Tracker, find the user story that you’re working on, and then mark it as “In Progress.” The user stories are typically single sentences such as “Put the top players per week on this page (refer to mockup)” and have points assigned to them by the engineering team beforehand. Then you discuss a couple of ideas, make sure you know what the state of play is, and start coding.

Only one person codes at a time. At this stage, I don’t know enough about the application to be effective on my own, so I’m mostly monitoring and following what my partner is doing. It’s an involving process to watch someone else code; you’re not just looking at the physical processes involved, but also the intent – usually I can catch bugs a couple of seconds after they’ve been typed, although I usually give feedback after we’ve reached a pause and I can be sure I’m not interrupting my partner’s flow. Actual disagreements are rare, partly because no-one is attached to an idea yet, partly because everyone is “on the stage” – you know that whatever you’re typing is going to be peer reviewed almost immediately, and it’s important to write something that’s going to be clear and acceptable to the partner at the time. I think that mixing pairs every day is also a factor here. Any time someone’s had a problem with code, it’s been new code that is unfamiliar to one person and not the other, so there’s a fair amount of learning that goes on each day.

It’s also draining. Reading code is almost as intensive as writing code, and there’s a ton of moving pieces, plus a couple of custom built frameworks that I’m unfamiliar with. More often than not, when someone asks me to take over or asks for suggestions, I have the odd experience of saying to refactor code using features that I’m not consciously aware of. I can say what to do at the time, and how to rework it, but when I get up from the computer, I have no clue what I just said or even what it meant. It’s a nice ability to have, but it’d be nicer if I actually knew how it worked.

The other reason it’s draining is because either you’re typing or you’re reading. There’s no downtime or distractions while you’re pairing. The “why” of this is interesting.

Grockit has email, but it’s mostly used for company broadcasts, housekeeping and customer feedback. It is not used intra-company, and this is a deliberate choice. Nearly everyone is in the same room. If you want to schedule something or start a group discussion, you mention it at startup and pull everyone in later. Likewise, there’s no instant messaging. No IRC channel. Nothing. So, no distractions. This took a while to get used to – I had to start scheduling and communicating with friends in one large block once I got home, and some people were even worried that I’d vanished completely.

So. You work until 1:30 pm, and then the lunch bell rings. Lunch is served in house, and is delicious organic vegetarian. After lunch, there’s a break where some people check their email on personal laptops, or play Rock Band, or run errands. People usually head back into the office at 2:30 pm, and then it’s pairing time again.

At this point, you’ve got most of the code written and you’re reworking and adding new tests to verify that what you’ve got actually works and that you can check it in. (Or the TeamCity build has broken, which results in immediate and productive feedback from the team.)

The test suite at Grockit is beautiful. Most of it is integration tests, with some mocks in place where they want to see specific exceptions. It makes heavy use of nested describe and contexts, and even has a very elegant use of should_behaves_like in the order processing logic that would have otherwise taken some complex metaprogramming of specs to do. There’s a strong bias to make tests useful and not repeat themselves, so tests usually only happen at one level. If the code can only be adequately tested using Selenium, for example, then there’s no point in writing unit tests for each method that would also test the happy flow.

Having said that, most of the tests involve business logic or concepts that I have to ignore because they’re not related to the problem at hand. Having to write or refactor tests while simultaneously not looking “over there” is mentally taxing, and then there’s the extra juggling involved with the mocking framework, selenium, and the assumptions made in the code itself. This is usually the point in time where my brain gets full. I have to pick up the threads of where we left off before lunch, so either I have to rummage around before I feel I know what’s happening again, or I just get a blinding headache trying to jump several steps ahead. Apparently, this is a common phenomenon during the first three weeks of coding, so I’m hoping it gets better soon.

At some point, the main user story is finished, tested, demoed, and handed it off to be accepted or rejected. Typically we get this wrapped up by 4:30 pm, and then pick off a couple of smaller user stories that involve front end work if they’re not too complex. This goes on until 6 pm, at which point the work day is over and everyone goes home.

And I mean, everyone goes home. There was one user story that I thought we could do, but was told “We can’t finish that; we only have 10 minutes left.” That’s discipline.

To recap: you show up at 9:15 am, do good solid work for eight hours, and then go home. You do not take your work home with you, or work on the weekends. User stories and estimates are scheduled so that all the work you can do is done in eight hours. Velocity is automatically tracked and measured using Pivotal Tracker on an eight hour basis. The hours you work is eight, and that’s the time you need to do a good job, with covering tests and code that checks the edge cases.

I like it. It’s not easy, but it’s so, so worth it to me.

I went to Patagonia for my vacation this year, cleverly timed for SXSW.

It was harder than I expected to unplug. For the first week, I had iPhone twitch (I’d left the phone deliberately). The second week it got easier as we got into longer and tougher hikes.

The food was good. The hiking was gorgeous, and I lucked out on my hiking companions, Lois and Kent (yes, really), who were and are charming people and an incredibly cute couple.

Photos are here: http://picasaweb.google.com/will.sargent/Patagonia

Check out the La Bomba Del Tiempo videos especially; 16 drummers all working together, beating up the crowd into a frenzy.

I’d tell you the blow by blow, but honestly going through holiday snaps is something best done on demand. If you’re interested in anything, send me an email and I’ll fill you in.

I’ve been going through an inordinate number of machines lately. I’m down one Seasonic power supply, two motherboards (one Megabyte, one MSI) and the Thinkpad came back from lenovo with the original problem solved, but with half the screen on the fritz.

This isn’t new or unusual; most of the machines I’ve put together have only lasted a couple of years or so. Consumer hardware just isn’t built for reliability; hard drives being the worst offenders. As a result, most of the information I really care about is based off-site. The two biggest exceptions are the iTunes library, and my financial data.

The standard solution is to have a backup hard drive. The problem here is that backup hard drives also go bad. I’ve run into this a couple of times with the Linkstation and the Western Digital External. There’s not much you can to do test this except restore from backup every once in a while and see if it actually worked (which is about as fun as it sounds).

Another possible solution is to use offsite backup such as Mozy or Amazon S3. I read Zawodny’s experiment with it and started using s3sync. But.

1) Uploading 30GB of music took a week.
2) Most, if not all, of that data was corrupted to the point of unintelligibility.

I only found this out when I thought I was missing some data, of course. Downloading 30GB of data is also a pain (and frankly, I didn’t expect so many errors – every single track sounded like it was playing underwater at half speed.

So. It turns out the best solution is to make sure you have an inordinate number of machines, and have them synced off the master. I may not be able to rely on one machine, or on a backup hard drive, but I can have music (or any encrypted data) put onto available laptops – and I know that data is good, because I play it from there. This is a looser implementation of jwz’s backup strategy, but it’s good enough for me.

Thinking about this a bit more, it’s surprising that backup technology is as limited as it is. Creating an encrypted bittorrent and sharing it amongst a pool would be an excellent way of ensuring redundancy and error correction (is this what Tor does?), but you may not even need to go that far; every time you do a backup, encrypt it, chop it up into yenc blocks, and dump it onto Usenet. A thousand servers will pass it around and make your data retrievable for all time.

The new company blog is up, and I can finally talk about some of the stuff I’ve been doing for months. Stealth mode, you can keep.

It is out of character for me to gush. But seriously, Topspin is the first company I’ve ever been in where I look forward to the meetings. Things just keep getting better every sprint. Our customers are literally rock stars. I’ve been a huge NIN fan since I was 18, and just getting the opportunity to help out was beyond awesome. It’s in Potero Hill right next to the Whole Foods, I bike to work every day, and every day there’s some DJing tunes through the Airport Express.

Oh, and the Billboard cover was nice.

Okay, I understand why Evernote is so cool now.

Take your digital camera and take photos of all your handwritten notes. Import them to the PC. Rotate them so that they’re the right way up. Drag them into Evernote. Wait for it to finish indexing. Presto. You now have all your notes in digital format no matter which computer you’re on, and you have them indexed and searchable.

Today, my brother and I found a man lying in the street on Van Ness and Market. His eyes were open but he wasn’t moving, and he was clearly unconscious. I checked he had a pulse and was breathing, phoned 911, and tried to figure out what the hell happened.

Pupils weren’t dilated. He had blood in his mouth, and it looked like he’d bitten his tongue. The back of his head was bloody. His heart rate was elevated, and his breathing was ragged – it sounded like his tongue was rattling in the back of his throat every time he breathed. It looked like he’d just been walking down the street, bit his tongue really hard and then fell straight back onto the pavement.

He started to move around a little bit after four minutes, and after five could say ‘police’ and ‘wanna get up’. I didn’t think it was a good idea for him to move until the paramedics came by, but I wasn’t going to hold him down either. It turned out he couldn’t really get up by himself, so we supported him until the paramedics came by.

I’m very glad he didn’t stop breathing. I was not looking forward to giving him the breath of life.

An hour later, we successfully contested a parking ticket before a hearing officer.