A notification popped up on my Slack messenger at work; it was from Mary, our office administrator.
Are you here today?
Now Mary has an interesting side career, she's a yoga instructor. But she is not a yoga instructor of the common, everyday, throw-your-leg-over-your-head variety. Rather she prefers to instruct in the more ancient and traditional notions of yoga - notions that include the physical practice but are also related to meditation and to philosophical detachment from the more selfish aspects of the ego. Given that, and the fact that her yoga practice had come up in conversation recently, I decided to poke at the question "Are you here today?":
Now that's a rather deep question don't you think?
Moments later Mary arrived at my desk with a copy of the Bhagavad Gita. Ha! I'd poked fun at her, but she was actually planning to lend me a copy of a book that delves into why this actually is a deep question.
This past weekend I took Baby Meg (3.5yrs old) on her first camping trip. And boy was it memorable. For starters check out our digs:
It's not a tent, and it looks an awful lot like an upscale deer blind. But no... as far as Meg is concerned, this is her "Castle in the Sky". I showed it to her in the afternoon and she was excited to find out that that night we were going to sleep in the Castle in the Sky.
On April 10th and 11th OpenSource Connections held their first (annual I hope) Haystack search relevance conference. It was intended to be a small-and-casual, 50-person conference but ended up pulling in roughly 120 people requiring OSC to scramble to find more space. The end result was one of the best conferences I've ever attended. In general, conference speakers have to aim their content at the lowest common denominator so they that they don't lose their audience. At this conference, the lowest common denominator was really high! So there was no need to over-explain the boring introductory topics. Instead the speakers were able to jump into interesting and deep content immediately.
Needless to say, I came away with a ton of good information that I'm going to put to work at Eventbrite as soon as possible.
Click tracking is a way of boosting documents based upon the historical clickthrough rate that they received when surfaced in search results. Here's how it works: Let's say that we're building click tracking for an online store and we want to boost the documents that are getting the most attention. First you set up logging so that you can count how times a particular item is clicked. Next you have a process that aggregates the clicks across, say, a week, and you store the value in a click_count field along side the documents that you are serving from search. Finally, when someone performs a search you boost the results according to the click_count so that items with high clickthrough rates start surfacing higher in search results. But if you think hard, there's a pretty nasty problem with this approach.
(Can you figure it out?)
The problem is feedback. In the context of search results, the first page, and really, the first few results get all the love. Very few users are desperate enough to click through to the second page of results. So click tracking causes a nasty positive feedback dynamic to arise: The user are shown a page of results, user's only click into those results, thus those first-page items now get an additional boost. This makes it even more likely for these items to show up on the first page of results for other related searches, which exacerbates the problem, etc. One way of addressing this problem is by tracking the typical clickthrough rate and then boosting a document according to only how much it exceeds the typical clickthrough rate.
This is the first in a series of blog posts where we will examine how a more sophisticated version of click tracking can be implemented and we will examine some of the neat off-shoots of this work that allow you to things like turning click logs into judgement lists. But first we start with a very simple example... a very simple example:
Today I had a Penny Chat with Will Acuff discussing how organizations can form relationships with communities. Will should know, he and his wife Tiffany founded Corner toCorner a group that made huge inroads into helping underprivileged communities in Nashville. The reason that I want to learn about this is because my church, (New Garden Church), is making a concerted effort right now to better connect to our community. In some ways we are positioned perfectly to do this - our church services are in Dupont Tyler Middle School. However we have yet to make meaningful relationships with the people in our community outside of our congregation. So we're looking for help!
Embedding spaces are quite trendy right now machine learning. With word2vec for example, you can create an embedding for words that is capable of capturing analogy. Given the input "man is to king as woman is to what?", a word2vec embedding can be used to correctly answer "queen". (Remarkable isn't it?) Embeddings like this can be used for a wide variety of different domains. For example, facial photos can be projected into an embedding space and for tasks of facial recognition. However I wonder if embeddings fall short in a domain that I am very near to - search. Consider the facial recognition task: Each face photo is converted into an N-dimensional vector where N is often rather high (hundreds of values). Given a sample photograph of a face, if you want to find all of the photos of that person then you have to search for all the photo vectors near to the sample photo's vector. But, due to the curse of dimensionality, very high dimensional embedding spaces are not amenable to data structure commonly used for spatial search, such as k-d trees.
As many of my friends know, I've picked up neuroscience as a sort of side hobby. (Some people collect stamps, I memorize anatomical structures of the brain.) Last time I blogged about this was regarding my Penny Chat with Stephen Bailey on his work with MRIs. But this week I sat down with one of Stephen's friends David Simon to talk about his research involving Electroencephalography a.k.a. EEG.
A week ago I met with an aspiring entrepreneur who had some interesting ideas regarding a recruitement startup. But during the conversation I got the feeling that he was holding his cards close and I was having a little trouble getting the whole picture. Towards the end of the conversation he confided that he was really vested in his ideas for the startup and that it actually hurt to hear those ideas criticized.
This got me thinking. I was also once this way - I held my "good ideas" close so they wouldn't be stolen away. And when people poked at the faults in my ideas... well, it hurt. But over the years I came to realize that this way of thinking is flawed.
I was lucky enough last week to find myself drinking a beer with Pat Poels, Eventbrite VP of engineering and two-time World Series of Poker bracelet winner. And I was luckier still that he was in the mood to talk about his poker days. I love hearing these stories but I'm always reluctant to ask because I suspect people ask him about "the poker days" all the time.
In the start of the discussion Pat was talking about just how much of an edge you have if you are able to read people closely. He told me a story about a very subtle tell that one of his old poker buddies fell victim to. This friend, we'll call him Bob, had the tendency to fold his hand quite predictably when confronted with the right circumstances. In particular, if Bob knew that he had a bad hand and if someone else showed an inclination to start betting aggressively, then Bob would quickly leave the hand on the table. Another one of Pat's buddies noticed this first. This buddy, (we'll say Steve), found out that every time he wanted to check on Bob's hand, all he had to do was riffle around his chips, indicating that he was in the mood to bet big. If Bob folded, then that would be the answer Steve was looking for. If Bob didn't fold then Steve would think long and hard about whether or not his hand was good enough to stay in the game. Thus, having this one insight into Bob's behavior and psychology provided quite an edge to Steve's game.
Following Pats story I aimed the conversation more towards business. "In the time since you retired from poker have you put your psychic ability to read people to any business use?" In my mind I pictured Pat in board meetings peering into colleagues eyes and inferring any hint of a hidden agenda.
"It's not a psychic ability." This is where Pat turned my thoughts around on me. "And the important lessons that I learned from poker were not about reading others but about reading myself." Then he explained: Even though reading others' tells could give you an advantage in poker and in life, that advantage pales in comparison to being able to understand your own intentions and motivations, your own abilities, and the limitations of what you can know about the world around you.
