baby steps towards commonplace

Transcript of the talk I gave on November 7th, 2024 at the Sensemaking Scenius group meetup

How many of you meditate? Alright. For those who don’t -

Bouncing DVD Logo (Source)

Do you remember the DVD logo that bounced around the screen, the one that appears after you press pause and wait long enough?

Now, recall the state that you were in. Do you remember finding yourself not so much engaged but held by it?

That feeling of being quietly held is a good place to start this talk.

Presenting commonplace … within an early version of commonplace.

I've been taking baby steps towards my digital journal project, commonplace, over the last couple of weeks, but this is the first time I'm presenting it to other human beings. In a sense, Commonplace is taking its baby steps in front of you all today.

Being online, for me, has felt like being within one of two spaces:

1. Spaces for Work, and

2. Spaces for Distraction.

Spaces for Work are where I can be Productive, in the broadest sense. Within them, I complete tasks that I have to do, resolve notifications that need my attention, and, over time, accomplish the so-called “Goals” I’ve set for myself. And these spaces - let’s call them tools - are designed with efficiency in mind.

When my tasks are completed or exhaustion hits, whichever comes first, the other place I can go to are places designed for Distraction, full of engaging, funny, weird content designed to keep me hooked.

Don’t get me wrong, I value being engaged by content. Much of my world - how I learned to love, my enduring fascination with Japan - comes from movies and books and music delivered through platforms designed to hold me in this state of engagement.

But being productive and being distracted aren’t the only modes of being we experience as full-bodied, living and breathing people. In some physical spaces, like a living room or a park, we can be there to simply exist.

And so, the question that commonplace is asking is if a digital space could be designed to evoke a mode of being that is not productive or distracted, just as a physical space does.

In other words: can we think and reason more as architects and filmmakers, and less like toolmakers? Can we make software that prioritizes feeling and experience? And can we design these experiences not for mass production, but for a few people who share our worldview?

But what is that mode of being which we’re trying to evoke, and whose worldview does it appeal to?

This is a recording of a pond in the National Taiwan University campus in early 2023, a pond that I would sit by and have some of my formative experiences with meditation. The fish in this pond - they're not koi, not carp, I don't know what they have been labelled and called. But I’ve observed them.

In the middle of writing my thesis and submitting assignments days-too-late, I’d take a moment to just watch the way that they moved - wandering, rubbing their sides against each other, glub-glubing at the surface of the water, their spines exhibiting a motion I can only describe as being “tensionless”.

In observing these fish, like the DVD logo a decade before them, I’d be brought to this state of stillness. To a quiet presence. Then to clarity. And, eventually, a deeper connection with myself, which made me more attuned to others around me.

It was this mode of contemplation, so different from my modes of anxious productivity or numb distraction I otherwise experienced online, where I found myself spending time. There's this quote by Naomi Klein,

“what haunts me is not exactly the absence of literal space, so much as a deep craving for metaphorical space, release, escape, some kind of open-ended freedom.”

commonplace is my attempt to construct this metaphorical digital space - a space made of symbols and signs, designed to facilitate stillness, presence and clarity. To open a space for contemplation, online.

But what is it, practically speaking?

It's a digital journal.

It won't replace your physical journal, because its designed with different intentions in mind.

One intention is to facilitate presence, currently supported by two design decisions.

First, there's a fish. Right now, it's a fish that swims in a circle, because it turns out that making a character sense its environment and determine its behavior in virtual space is pretty hard. It’s more video game programming than traditional software, video game programming is hard, but I'm learning.

My design goal for this fish (and future schools of fish) is this quality of “aliveness” - for it to be there with you, present, making its own decisions of how fast or slow it is to move, and how it should approach the breadcrumbs you offer, and whether it should eat it. If it is convincingly alive, then you’ll observe it, as I once did its Real counterparts in Taiwan, and it will bring you to stillness, as it did for me. And maybe, later, you can befriend it. But how to think about this digital being is for later, maybe in the next edition of this talk.

Second is the act of writing. I've been exploring different ways to manipulate text that support this feeling of presence, of giving you the freedom to "just write". I experimented with No Backspace, disabling the ability to edit text once it has been written. This was as fine as it was frustrating, so I relaxed the approach, and this is what I’ve discovered: a mode where you can choose to just attend to the single word in front of you, isolated from past or future, dimming the possibility of the thought that your writing isn’t good enough.

This change to text editing is subtle, but its effect on the psychodynamics of writing, personally, have been profound. I’ve found my writing come in tune with how I think, perhaps because it’s hard to pass judgment on the quality of one’s writing when all that you can see is a single word. And it makes me want to deepen this by, for example, alternating between writing into it and speaking into it, with my voice preserved and transcribed inline.

Now, for connection. It's probably not advisable to take a page out of my actual journal for a public demo, but this one touches my heart, and I don’t think I’ll be transcribing this and posting it publicly (addendum: lol, hello, dear reader). From an entry in October, this is me reflecting on how,

"my friend Pablo sends me pictures of the sky every day on his walk back from work. We're staying in touch this way. Here is my sky and here is yours."

Journals, we think of them as containers that we pour ourselves into. But an alternative metaphor I'm experimenting with is that of a resonance field, where we can locate ourselves alongside others who are thinking or experiencing something similar.

For example, Pablo and I were paying attention to the sky together, forming our shared experience. The system picked this entry up and placed it in a field of similar experiences, pulling out a "sky above you" channel, where people have shared images, quotes, and thoughts from their own worlds about the sky. Even if I don’t contribute to it, which the Chat view on the channel nudges me to, our personal and private experience has suddenly felt ... common. Me and Pablo, and strangers from around the world at different times, have paid attention to roughly the same thing, and we can, in later versions, perhaps connect around it. My journal stays private, but it also becomes social.

Buddhist Directions for Interactive Systems

With fixes and polish, my vision ends, and where it goes thereafter is ... wherever it goes. Which is to say I don’t have “a vision” that I’m “building” as much as I have a seed that I’m watering. I’m gardening, and here is where I see commonplace growing.

First, I'm trying to deepen this metaphor of spaciousness, losing lines and rigidityin favor of a gestural, almost crayon-like feel. So no modes and tools that you switch in and out of like on a canvas, but more like if you draw a box the system will recognize it as a command to open a portal and a squiggle as another.

Another direction I'm curious about is constructing a person’s “World Model” using their journal entries. In machine learning, a World Model is a representation of the world—of the things within it and how they relate—constructed by a machine learning model observing input data, enabling predictions about future states.

What strikes me about this concept is how naturally it also applies to humans and the ways we model our worlds. Except, in our case, it seems we construct less of our world through direct observation and more through our consumption of media—something we have surprisingly little control over. Maybe that's why, during my shrooms trip, I found myself unexpectedly thinking about Sam Altman’s reported ayahuasca journey :“Sam Altman... Why are you here? How did you get here? I have no idea who you are! Get out of my trip!”. All this to say, if you can model someone’s World, you can facilitate things like media / knowledge discovery and possibly a so-called “aligned” recommender system (with recommendations aligned to your values, as modeled in your World Model).

An exploration that came out of World Model was to treat writing as an opponent process between you, thinking as you do, and your shadow, thinking in your contrast. So this is a joke to form an example, but say I wrote “I wish I was a fish”, my shadow replies with “I'm content with who I am”, and I can either escape my shadow by pressing Escape, or adopt it by pressing Tab, and steer my thinking in its direction. Full credit, here, to Svitlana for both introducing the Jungian notion of a Shadow to me, and seeding this idea of shaping your inner voice through interactive writing. Taking a step back, commonplace would not exist without her encouragement, wisdom and creative partnership. This product with personality is as much mine as it hers.

products with personality

Another is knowledge discovery, which I think appeals to the sense-making perspective we share here. With LLMs, we have the ability to extract concepts from media at runtime and, through a clever combination of semantic search and graph traversal, relate it to shared concepts and media, without losing the underlying semantics of “your” concepts.

One interaction pattern this opens up is this: oftentimes, we use everything but the word to describe what we are looking for, and that tends to show up just as we're writing or thinking. For example, its pretty plain to see that I was Anxiously-Attached and Existential last fall by interpreting my journals, but I had no knowledge of these concepts and the knowledge that they would have unlocked, knowledge that would have, quite frankly, not just been useful or interesting to me but might have altered how I lived my life. So you could imagine using your journal as a starting point for a knowledge discovery journey, wherein you discover the concepts first from what you've written, and then you unlock a journey of discovery of concepts and media from it.

This is it. Thank you.

Audience Questions

Read if curious, this hasn’t been edited. Names have been anonymized and numbered as audience member #A1/2/3/4/5. My initials, AV, are preserved.

#A1: Could you just walk through that last example again? What is happening here? I'm not sure I follow you.

AV: This is going to be a confession, but if you read my journal entries from November, you would say that I was very existential, but I didn't have that word. So it's very easy to look at something and be like, this is the thing which can unlock a deeper exploration. The thing is that those concepts which you're discovering are also connected to media, in Arena already, once they've been extracted. So you can use that concepts to find media connected to it. So it's not just like... You can just say existentialism and then write up a Wikipedia, but then you could also, like what you guys at SenseNets are doing, show the media that actually speaks to it, and have this social context around it.

#A1: Yeah, that's really interesting. I feel like it reminds me of that feeling when you learn that there's a word that evokes some experience. You only have the gut feeling, but someone tells you, oh, that's actually this. And then you're like, oh. And then you find the rabbit hole, like, oh, there's a bunch of people feeling this. And it's like, well, cool.

AV: Especially in ways of knowing that center, is it phenomenological experiences? You generally are doing the thing before you find the term in the space that you're doing the thing.

#A2: This is really, really cool. I like just how mindful you can be when you're writing. I guess the thing that I'm thinking about is like, is there also a way to zoom out and just see it as a doc, like a Word doc or something like that?

AV: I've been thinking a lot about the high-level view. There's this sense that if you can take a high-level view on someone's journal, what you're really looking at is how they're changing over time, and how their world, so to speak, is changing over time. And I really like that there's... So when you do these explorations, and so each journal entry is a space. Things that you find in that space, they sort of live with you. You discover them and become a part of you. I'm really curious to see how that changes, how you can see how that changes over time, and how you can use that for things like monitoring personal growth, even? I don't know, but that's a little abstract.

#A2: It's something that, #A4, this might be really relevant to personal knowledge. The PKMs, right? Obsidian and the tools there. Some people really like that explicit structure, like double brackets, piece of content. And then there's, oh, okay, maybe you do some sort of fancy vector embedding stuff, and have some AI facilitation of how your journals relate to each other, and how they relate to you at any given moment. So there's some interesting space to explore. Depending on how people want to use this for. Because I feel like you can get feedback from a lot of people. As a writer, sometimes there's almost anxiety from the previous word or the previous sentence. And I think that's something that the interface itself is mitigating.

#A3: How do you see privacy?

AV: This actually doesn't have a database right now. I intend to use this jazz.tools because its local first and encrypted, so you own your data, its private to you, and the app is fast.

#A3: Do you know who you're relating to, or do you decide what you share with others?

AV: I think having the choice to share, at multiple levels of access control, is crucial. It will be on the interface. As to the people that you're seeing on Arena, these are public channels already.

#A3: Yeah, but the concepts? So you see other people with similar concepts, and you can set or define, right?

#A1: No, but they're not sending their journals.

AV: Yeah, it's not their journals.

#A3: It's not their journals?

AV: These are just media. These are just images and videos stored in arena channels and indexed by commonplace. There's no journal-based matching mechanism, yet, a “find people like me”.

One mechanism that does this, though, is where somebody who's going through a similar thing as you is brought up, and then the consent mechanism is that you're both anonymized until you both agree to reveal. But that's also a network effects that suffers immensely from the cold-start problem, so I’m focusing on the experience of writing, first.

#A3: One of the guys living here on this office is working on some kind of cryptographic pairing thing like that.

#A4: How would you imagine getting to commonplace? You gave us the story of the work life and then the distraction media life. Where's the intervention? The timer thing on your phone that's like, you've been scrolling for this long. You've been working, and then it brings you here. How do you see it?

AV: I never thought about that, actually. Maybe you just have a... it just calls to you. You just want to go hang out there. That's maybe what I want to design. Yes, but also, I don't know, personally, sometimes I know when I need it, but sometimes I don't know. I feel like I don't like those type of things. I don't use them.

Maybe it's the fish and you're somewhere, and it swims to you and reminds you.

AV: Maybe. This is what I like about the home-cooked app philosophy, it gives you persmission to trust your gut and make design choices. You don't need to have it be so engaging.

#A4:: It's not responding to a market necessarily?

AV: It's not responding to a market. I'm making it primarily for myself right now. It gives me an excuse to work with rendering engines and learn how to write shader programs, which are amazing because this can run on the web at 60 frames a second on your phone when it's at 10% charge, and take 100ms to load.

If there's a few people around me who are touched by it, they can use it too. I really like that philosophy for software as well. I was in the DeSci space for a while and then worked on a Twitter client. All of these things had grand ambitions, and it feels nice to work on something small.

#A3: I've always seen this as the coolest screensaver default. I'm going to stare at this. It's not what takes you to that moment. It's always there.

#A1: It's like a screensaver, but then you can call it “sanitysaver”. It's not just saving your screen, you can go there to save your sanity.

AV: I'm very, very excited about actually having the fish feel like a being that can just move in its own ways. The thing in my head has always been you can throw a bread crumb and it will go and swim and eat it.

#A3: It's like you're doom scrolling and then suddenly the fish is taking you away from doom scrolling.

AV: The fish is a symbol. It was a symbol for me in Taiwan, too.

#A4: It can get sophisticated if you start bringing biometrics. This guy is getting stressed. The fish comes out of nowhere. What if the computer operating system is like that? You open your computer. It's blank. There's a fish. You can write something up. The right things are going to pop up for you. What if it's a computer? You can go for a walk in the forest. He's like, I'm not trying to make this ambitious. What if you just make it the whole time?

#A4: On the other hand, I think some of the things that you post here are actually quite intense, honestly. The shadow kind of interaction. I could imagine. I journal a lot. Some of that stuff feels really raw, really real. It's not fun to go there. Of course, there's a lot of peaceful, calm energy here, but I think there's also a really deep self-knowing vibe that isn't very calming at all. I encourage you to explore all of that. I really like what you've done, even just with the, I wish I was a fish, but kind of giving people a playground of possible experiences of self-knowing, from the really calm to the take something really vulnerable that you wrote in your journal and see what that means on the opposite side of that.

AV: With the shadow work path, the tone changes. And I’ve explored this self-knowledge path a bit more. Another experiment, I ran this with Svitlana, was we were experimenting with personal belief networks. A personal belief is just an I am statement about who you are. I am kind, for example. Exposing this whole network gives you a picture around a person's personality or sense of self. It turns out that you don't really need a personality quiz to get the network. We could talk about the ethics of this, but then just processing it with a language model is enough, and that could be a useful tool for self-analysis, but the tone changes.

I'm still figuring out the tone. Is this just a playful space where you can just be and then write and then occasionally find other people who are thinking the same way, or is it more about how do you construct or analyze or deconstruct the Self in the therapeutic or clinical sense?

#A4: I love the playful. It's a playful work here.

#A5: Thank you. It's really interesting. When you're incorporating an LLM to your software, it's like you're almost playing the role of director of a movie. You have to figure out what the character is. It's a funny observation. You're kind of talking about, well, I have to get the tone right here. What's the relationship? What's their vibe? You're working on software, but you're almost trying to figure out what it is as a character in the play that's your software.

AV: It's 100% experiential value before it is utility, which is also why I said the question I have, the question I'm trying to answer versus the problem I'm trying to solve in the talk.

#A5: Actually, the problem you're trying to solve, I feel like that framing, that's the thing that's almost hard. The space of where you go with this is so huge. You focus, at least in the beginning, on the work and how we get out of that work zone.

AV: It is. And I’ve barely scratched the surface. The web is so expressive, yet so untapped. This loads in 100 milliseconds. Even when we add the living behavior, it's going to continue to be very fast - a game that can load in an instant.

The web is expressive, but we still put software into boxes. It's capable of being more, especially when you combine rendering or game engines with streaming-based data architectures, which you couldn't really do when you were thinking in terms of HTTP and DOM rectangles.