An Accidental Universe

For about six months before I started building Skysquare on Tap, I had been running a much simpler system against the Bluesky Jetstream.

Because I obviously could not store everything happening across the network, I had started with a registry: several thousand domains that I thought represented a reasonably broad cross-section of public discourse on the web. It leaned heavily toward the canonical newspapers, magazines, broadcasters, journals, and other publications that tend to circulate through the news and ideas ecosystem. I assembled it partly from existing resources I found online and partly through my own judgment about what belonged there.

There was nothing particularly scientific about the list. It reflected a series of choices about what kinds of publications mattered to the application, and like any list of that kind it necessarily excluded a great deal. I also assumed, somewhat casually, that I could simply expand it whenever I wanted to in the future—something that seemed straightforward enough at the time, but which I am currently discovering is rather more complicated than I expected. That is a subject for another post.

At this point, I simply needed some kind of boundary around what I had begun thinking of as the “Skysquare universe.” Skysquare was not intended to archive everything people did on Bluesky. What I cared about were the moments when conversation on Bluesky reached outward toward the web—when somebody shared an article, essay, report, interview, or some other piece of source material that Skysquare might eventually be able to connect back to the discussion surrounding it.

So the first ingestion system did something relatively straightforward. It watched Jetstream for new posts, looked for outbound links, checked their domains against the registry, and retained the activity that fell within the universe I had defined.

At the time, I thought of the registry mostly as a practical constraint on the amount of data I was collecting. What I did not appreciate until much later was that six months of running the system this way had also produced another dataset, almost by accident.

Every qualifying post also identified the person who had shared it. In AT Protocol, that identity is represented by a DID, or decentralized identifier. A person's handle can change, but the DID remains the durable identifier underneath it. And so by the time I began seriously considering Tap, I had accumulated not only six months of links and posts, but a large collection of DIDs belonging to people who had demonstrated a very particular kind of behavior: at some point during those six months, they had shared something from one of the several thousand publications Skysquare was watching.

That turned out to be an extraordinarily useful accident, because Tap approaches AT Protocol data from a rather different direction.

Jetstream had encouraged me to think primarily in terms of events. A post appears on the network, my application inspects it, and if the post contains something Skysquare cares about, I keep it. Tap is much more naturally organized around repositories. Every account on AT Protocol has a repository associated with its DID and hosted on a Personal Data Server, or PDS. Instead of defining the system entirely around individual posts as they passed through a stream, I could begin defining a population of accounts whose repositories Skysquare wanted to keep synchronized.

This was an important shift in the way I was thinking about the problem. Jetstream gave me a way to see relevant activity as it happened. Tap gave me a way to organize the ongoing system around persistent identities rather than isolated events. Recovering the history that existed before Skysquare began following a particular account still required additional infrastructure, but Tap provided the organizing principle I had been missing: once I decided that a DID mattered to Skysquare, I could treat that repository as part of a population whose past and future activity the system would continue to understand.

That immediately forced me to answer a question I had not really needed to answer before: whose repositories did I actually want Skysquare to follow? I could have tried to make the answer something approximating everybody. But by this point I was already beginning to understand that the fact that an application can collect some piece of public data is not, by itself, a particularly good reason to collect it. More importantly, Skysquare did not need a complete replica of Bluesky. The application had a much narrower purpose.

And, conveniently, the previous six months had already given me a population that seemed reasonably well aligned with that purpose.

The people in my existing dataset had all crossed the same threshold: they had shared a link from the public web, and not merely any link, but one from the broad universe of publications Skysquare had been monitoring. That did not make this population statistically representative of Bluesky, and I do not want to pretend otherwise. It inherited every limitation and bias embedded in the registry that produced it. Somebody could spend all day participating in serious political or cultural discussion on Bluesky without ever sharing a link from one of those domains. Someone else might primarily share small independent publications that I had simply failed to include.

But the boundary made sense in relation to the application I was actually trying to build. Skysquare was never intended to become a mirror of Bluesky. It is intended to understand the relationship between conversation on Bluesky and source material on the open web. If someone had already demonstrated that they participated in that relationship by sharing one of the publications Skysquare monitored, then including that account in the population the system followed seemed like a defensible criterion.

So I made what, in retrospect, became one of the more fundamental architectural decisions in the project: I used the DIDs I had accumulated during those first six months as the starting population for the new system.

What had previously been a filter on individual posts now became something closer to a behavioral threshold for inclusion. Under the Jetstream architecture, Skysquare saw a qualifying post and retained that event. Under the new architecture, the same post could tell me that the account behind it belonged within the population Skysquare was interested in understanding over time. The person only had to cross that threshold once; the DID gave the system a persistent identity around which both past and future activity could be organized.

I liked that criterion precisely because it was behavioral. I was not trying to construct a list based on follower counts, verification, celebrity, professional status, or whether someone had ever heard of Skysquare. The relevant fact was simply that this public account had participated in the movement of information between Bluesky and the portion of the open web the application was built to understand.

There was, however, an obvious problem with using my existing dataset as the starting population. The six months of Jetstream history could tell me which DIDs had already crossed the boundary, but Bluesky was not going to stop changing on the day I changed the architecture. New people were joining the network. Existing accounts that had never previously shared one of the publications in my registry could do so tomorrow. If I simply froze the population I already had, the portion of Bluesky Skysquare understood would immediately begin going stale.

So Jetstream remained part of the architecture, but its role changed. Rather than being the primary way Skysquare understood every relevant account indefinitely, it became a way of continuing to discover new people whose public activity intersected with the universe I had defined. The broader network could continue producing new qualifying accounts, and those accounts could then become part of the persistent population Skysquare followed.

That gave Skysquare a way to keep discovering new qualifying accounts as Bluesky conversation continued to unfold. It also began changing what the data itself meant. When I began the project, a Bluesky post was mostly useful because it pointed to an article. I wanted to know that somebody had shared a page, what they had said about it, and whether some portion of the article itself had been quoted. But once Skysquare began organizing data around persistent identities rather than treating every post as a completely independent event, the person behind the share became much more than metadata attached to a URL.

Over time, the system could begin to see patterns. Which publications did this person repeatedly share? Where did the same people appear around different articles? Who tended to encounter a piece of writing early? How did different portions of this population overlap around different sources? The original product idea was still there—connect the conversation back to the article—but the infrastructure was beginning to make a much richer view of the information ecosystem possible.

I had created the registry simply to make the Jetstream problem small enough that I could work with it. But that temporary constraint had quietly become something much more consequential: a definition of the social universe the product was capable of understanding. What had initially looked like infrastructure plumbing had become a product decision, and the product decision in turn encoded a particular way of looking at the relationship between Bluesky and the public web.

I was quite happy with that architecture, and in many respects I still am. What I had not yet fully internalized was the arithmetic hiding inside it.

Every day, the system could discover more qualifying accounts, and every new account potentially brought both historical activity and an ongoing stream of future activity into the system. Under the original Jetstream model, growth mostly meant another relevant post appearing in the stream. I had now moved toward an architecture in which a single relevant post could make an entire additional identity relevant to Skysquare over time.

But the larger implication was hidden in the registry itself. At the beginning, adding another publication had seemed trivial: it was simply another domain for the system to recognize. Once sharing from those domains became the behavioral threshold for entering the population Skysquare followed, expanding the registry meant something much larger. A new domain could reveal an entirely new population of qualifying accounts, together with activity the system had never previously considered part of its universe.

The registry was therefore no longer simply determining which links Skysquare recognized. It was helping determine which portion of Bluesky Skysquare could see. Expanding the web Skysquare understood also expanded the population of Bluesky it had to understand—a much more consequential relationship than I had appreciated when I assembled that first list, and one that would prove considerably harder to change after the fact.