You made a wrong turn along the way. Maybe the map was upside down. You find yourself alone in a wilderness... that is, aside from hidden pits that make you ever vigilant; aside from tangling vines that impede your progress; aside from lonesome howling winds; and aside from the silence – the absence of the only voice you really want to hear. There is a light at the distant horizon, and you must reach it ... or maybe it was a mirage. – Why, oh why, couldn't you have been one of those crystal rubbing hippies? At least they act like they've figured it out. At least they seem... at peace?
There's a lot of amazing things about how the brain works, and in this post I'm going to explore the phenomenon of binaural hearing – that is, your brain's amazing ability to hear with both ears ("binaural"), combine the signals, and quickly extract useful information in a way that I think should basically be impossible for the slow meat-computer that it is. Crazy stuff! Here we go!
I wrote the first draft of this in roughly 2017 but left lots of details out. I happened to run across this again about 2 weeks ago and it now seems quite apropos for the time we're living it, so I spruced it up a bit, added some of the missing details, and here I'm publishing it. Enjoy!
I know it has to be true – The magic of human-level cognition isn't the result of a brain in which every single piece is perfectly tuned for its very special and isolated task. Instead, there has to be some simple "principle of computation" that is repeated throughout the brain; some component that, when replicated throughout the brain, gives rise to this emergent property we call intelligence. Sure, we have a motor cortex, specialized to control movement. We have a visual cortex, specialized to process information from our eyes. And we have a pre-frontal cortex that assimilates all this information together and plans what to do next – "That's a snake! Don't step on it!" But there is evidence that the base circuitry that makes up all these modules is actually quite general. At every point on the neocortex, you see basically the same pattern - an interwoven computational "component" composed of 6 layers of stacked neurons. At every portion of our cortex this pattern is, with very little modification, repeated. Besides the similar morphology, there is other evidence that the computational components are general. In several really weird brain rewiring studies, they have redirected visual input to the auditory pathway and shown that animals can compensate quite well - effectively "seeing" with the part of their brain that was rightfully intended to hear stuff! (Mice (Lyckman et al., 2001), ferrets (Sur et al., 1988; Roe et al., 1990; Roe et al., 1992; Roe et al., 1993; Sharma et al., 2000), and hamsters (Schneider, 1973; Kalil & Schneider, 1975; Frost, 1982; Frost & Metin, 1985).)
Over my half-career in software development, I've started to collect some insights (or at least opinions) about how software can be built so that it is easy to maintain, use, and extend. Usually we hear of principles such as modularity, abstraction, loose coupling, and separation of concerns, and each of these is important to strive for. But I've found that behind all of these, there is a single, unifying principle – the reduction of cognitive load. In this post I talk about what I've come to think of as the primary objective of software design: minimizing total cognitive load of all future users and maintainers of your software.
Vector search has taken the world by storm. The idea is this - cast documents into a vector embedding space via some transformation (e.g. BERT) and then index them into a vector database that is capable of fast approximate nearest neighbors search. Then when the user makes a text query, cast their query into this same embedding space and find the nearest vectors to the query vector and return the corresponding documents.
It occurred to me today while staring at the sky blankly, that this type of search won't work when searching for grouped content. For instance, what if I go onto Reddit, and I'm not really looking for a single reddit page, but I'm looking for an entire subreddit that matches my interests. Here I'm not looking a document, but a group of documents that match my intent.
I realized today that I don't write blog posts most of the time. If you were to look at my private notes, for every blog post that I've published here, I have probaby 10-20 half baked ideas that would be great to write about... but I just don't have time to get around to them. Why not? Well, because frankly, writing a blog post is a time risk. If I determine myself to write a post on some Friday afternoon, it's very possible that I could be signing up for a whole-weekend work task, or even more if it's really a good post. Some good posts require research and creating proofs. All good posts require lots of time for actually crafting the text and refining it. What's more, blogging is a rather lonely task. All this preparation I do alone, and that takes me away from time I could be spending around people – which I much prefer.
A quick post tonight – just relating an interesting observation that I made about train whistles.
I set outside and work any time the weather is fair. And in doing so I often hear one of the local trains in the distance blowing their whistle as they cross at an intersection. I realized one day that if you listen carefully, you can tell which direction they're going – toward you or away from you – according to the sound of their whistle. Now obviously, there's the Doppler effect, so that the sound of their whistle is higher pitch if the train is coming towards you and the lower pitch if the train is going away from you. But the Doppler effect by itself isn't that helpful because you (or I at least) don't have perfect pitch and don't know what the normal pitch of a train whistle should be.
(Note to reader: I think I wrote this post for myself. From an outside perspective, it's by far the most boring one I've ever written. But it's math that's been occupying my mind for a week and from an inside perspective it's been quite fun. Maybe you'll find the fun in it that I did.)
For a project I'm currently working on at GitHub, I ran into my first statistical distribution defined on rational numbers and I found it weird and interesting when compared to the continuous distributions that I'm used to. We were looking at feature adoption on a per-repository basis. We defined adoption to be
\[\textrm{adoption}=\frac{\textrm{number of people using feature in a given repo}}{\textrm{number of people active in that repo}}\]
The question is, what should the distribution of feature adoption look like across all repos? Pause here and think about it. (Don't even scroll down!) Leaning upon my normal intuition with continuous distributions I was initially a bit surprised with what I found.
