Anthropology Math

There is a natural progression in building innovative news. After having the idea for more than 25 years expressed as a domain, eigentimes.com, I've built Eigen Times! Embedding the news as eigen vectors in teh eigenbasis of all news ever. First with The Guardian and Wikinews for about 25 years, and then extended it to 175 years of the New York Times. The v2 had to prevent the NYT from overwhelming the present with its old obituaries, so a rebalancing was needed. Then I added Eigen Hacks, processing Hacker News. That required a balance between the original link and the discussions. We extended our algorithms further.

Now we have eigen news for every day, with every news story aligned against an eigen vector of all news:

eigentimes.com

eigenhacks.com

What's missing are the people -- or, rather, a people-centric view. And here I also had the idea, expressed in my love language of ideas, as a domain anthropolo.gy. Build a social graph of all things interwoven in Silicon Valley -- founders, tech, projects, funding, events, talks, videos, photos.

We built anthropolo.gy as people vectors! A person can be embedded in the eigenbasis of all people, and eigenvectors will be the representative archetypes, like a database CEO (probably very much aligned with Larry Ellison.)

The idea of eigen times was conceptually clear for me. But the math, as implemented by Fable overnight at Penn, turned out to be quite advanced. I went to visit my PhD committee member and the father of Corpus Linguistics Prof. Mark Liberman, a dorm advisor at the Quad, and met with the students figuring out careers in AI era. Then I visited Mark in his dorm apartment and we talked about the early days of Bell Labs, where Mark worked alongside the pioneers of Unix and digital news. I came home to my West Lofts apartment inspired to build Eigen Times with Fable, and we did. It worked overnight and built it all. In the morning, I connected my AWS account, and we published it in an S3 bucket.

Fable has made many optimizations and choices to the basic Eigen math, and I asked it to write a paper about Eigen Times, which we did together, iterating on clarity. The math turned out to be much more, with modern optimizations, and I asked Fable to write a math companion for the main paper, Eigen Times Math, with illustrations and worked examples.

When we created Anthropology with Astra, I asked it to do the same -- the main paper and the math companion for it. Encoding and decoding people as vectors over time required windowed news aggregation per person, with relationships for graph construction, which entailed more math.

And then I wanted to work through the examples myself, and asked Asra to generate notebooks in Python and OCaml, and spin up a Jupyter server so I could play with them from my iPhone over Tailscale. The default layout did not work well on the phone, so we built a Jupyter extension for mobile work.

You can't know where you are going if you don't know where you've been. History provides starting conditions for a trajectory landing somewhere. My classic Soviet math education built on the arc of great mathematicians: Hadamard, Legendre, Gauss, Euler. I asked Astra to write a history and literature review, introducing foundational concepts used in our work as they first appeared, springing from the minds of their creators on paper.

The Eigen Math History paper turned out to be quite long, dense, and interesting. It also added new ideas that connected those used in building our newspapers. So I asked for a math companion too, called Eigen History Math (instead of Eigen Math History Math).

And that added two more notebooks.

When reading my new papers, I found many obstacles to understanding. Some of the variables in equations were not introduced, or relied on an implicit convention that was not always clear. We did a pass over all papers to ensure that every single concept used in an equation is introduced beforehand. We added tables and glossaries and indices.

Reading the properly defined papers was still a rollercoaster ride. Some passages went over simple definitions slowly and deliberately, while others compressed new ideas into a few lines and introduced new concepts and methods while packing them into a paragraph.

So I asked my AI companions to do another pass and expand any compressed presentation into a piecemeal one. That reminded me of my pattern of study at Dartmouth and Penn, when I'd stare at a formula without knowing what some of its parts were, while the professor forged ahead. I often wanted to go over an example, or write some code to understand how it worked. Now with First Pair, I can ask AI to explain the math better while also having it write the code that will reify and run it. And the notebook format allows for the whole paper and book to become executable!

The notebook adds another delivery format to First Pair, for math explainers.

And after I've done all of this, I feel like I understood something after all these years back at Penn, with my own AI-aided research. I feel that I own my knowledge now and the process of understanding and acquiring it. If I am one of those people who learn better by coding and running, rather than looking at formulæ, I can use it to accelerate my understanding. There's a pedagogical method here, worth generalizing.

So we've written one more paper on learning with AI, Personalized Computational Pedagogy.

All of them are available at First Press:

Eigen Times paper Anthropology paper Eigen Math History paper

-- and their math companions: Eigen Times Math Anthropology Math Eigen History Math

Personalized Pedagogy