Kinkakuji and Ryoanji (11080372334) [commons]
In 2022 I took a couple of months off to focus on math. By profession, I am a software engineer. But I am also very curious about the Philosophy of Mind and Artificial General Intelligence (at least I was back then). Driven by these interests, I wanted to learn how to build neural networks from scratch,[1] which led me to relearn Calculus and learn Linear Algebra and Probability Theory. This plus some Python books on deep learning got me going on this front, and I used some of my time within those “free” months to firm up what I’d learned over the previous couple of years.
But then I decided to take a step back and learn some “real math,” which I grew more and more interested in. Which was, for me: proof techniques. From which I veered a bit into Group Theory, and a liiittle bit into Category Theory. The background here came from my interest in Functional Programming. Which you can pick up without any foundation in Category Theory. But some of the concepts are in fact theoretically grounded in Category Theory.[2]
Something about Category Theory fascinated me, but I could never wrap my head around it. To backtrack a little more here, what got me “on the hook” initially was something very silly, namely the name “Category.” At that time I was fascinated by the concepts of “Categories and Concepts” in Cognitive Science[3] and Philosophy of Mind, because I found those to be the keys to understanding our minds, in order to build Artificial General Intelligence. I thought the concept of “Core Concepts” (as promoted by Susan Carey)[4] to be a more scientifically grounded version of Kant’s “Categories.”[5] And by the logic of David Deutsch, we would have to understand the mind before we could build one.[6] Kant showed that the mind imposes structure on the world.[7] So the question was, what does this structure look like? (We won’t explore this here.)
In any case, with Category Theory, I didn’t really “get it.” There was on the one hand this inscrutability of many things, but on the other hand this, what I thought deceptive, simplicity. In the last couple of days, I discovered that some simplicity is in fact real simplicity. The core ideas of Category Theory are in fact very simple. And they are the most abstract and general things in mathematics there can be, just like the Philosophical Categories are the simplest structures there are—by definition; because everything else is structured by them.[8]
When I watched a couple of videos on Category Theory, somehow a few things fell into place for me. One of the videos talked about how there exist functions between the Grp category and the Set category and that one of those is the “forgetful functor,” which makes the group properties “go away.” I believe what that basically says is that all groups can be mapped to sets and thereby the group-specific extras disappear. That also means different groups share the same underlying set. By implication, from the set, the groups are not recoverable.
Where things fell into place here was that back then I had worked my way partially through _An Invitation to Applied Category Theory: Seven Sketches in Compositionality_[9] which I found amazing, although I had the feeling that I didn’t really grasp things, even if I could solve quite a lot of the exercises. But I think the main point of the book was to give a practical intuition that structures can map onto other structures, and sometimes parts of those structures cannot be captured in the target structures, and sometimes they can—in which case we can talk about structure-preserving maps.
This is a very interesting thing. I was reminded that I read at some point that Category Theory is the discipline which lets one basically decide whether the data of one database can be captured in another database[10]—one with a different schema. It can (I think) systematically answer the question of how much information loss to expect when performing a schema mapping.[11]
I am by no means an expert in math. My contact with these topics these days is driven merely by an interest in understanding certain philosophical things which have a bearing on modelling, particularly modelling ‘knowledge.’ And here, for me, there is an open conundrum. That of how to make a case that knowledge about reality—not reality itself—is necessarily limited. Or, any set of consistent ‘facts’ (if you will) is limited in size. But different mutually contradictory but internally consistent sets of facts can together yield a fuller picture of reality.
This is all my intuition, which I found repeated in the depiction of the Zen Garden, as brought up by Iain McGilchrist in The Matter With Things. He is talking[12] about the Ryōan-ji Rock Garden. From whichever standpoint you look in that garden, you will never be able to see all rocks at the same time, as some of them are inevitably occluded by others. Whenever I think about this kind of thing I hear echoes of Gödel’s theorem. But however often I hear explanations of it addressed at (semi-)laymen like me, I feel I don’t understand it except at too intuitive a level for me to feel comfortable adducing it as a real argument here in support of my case.
Now, the common intuition is that all knowledge can be captured in a coherent and consistent facts-base.[13] I see this intuition reflected in the concept of the scientific method itself, but also the shape of the book The Beginning of Infinity by David Deutsch, which paints a picture that we forever approach reality with our knowledge, with that reality itself being the limiting case. This is different in spirit from the picture of the Zen garden, which tells me such convergence is only locally[14] possible.
In practical terms, I have worked in two very different environments. One was in academia, where we, for example for archaeologists, developed different databases with the goal of capturing knowledge. One such database was extremely practical, namely that of capturing archaeological field data.[15] The limitation in modelling we faced was that there was hardly anything any two teams would agree on in terms of data model. So there was a huge need for great generality (and then great configurability). I think where such needs come from is that in such databases the questions one wants to pose to the system are not known beforehand, so the data cannot be structured, because that structure is structure for answering some question.
The other environment I work in is that of enterprise software engineering. Here, the data model (commonly called the domain model)[16] is generally simple. This is, as a corollary to what I’ve said before, because we know the “questions” beforehand, as these are precisely what the requirements tell us the systems should be able to “answer.”
My argument now, or evidence, to further my case that knowledge is not universally modellable, in a way that we can capture all ‘knowledge’ in a single database, is that very obviously there should be infinite demand for interoperability between those systems, as that would enable many many use cases. Where is that interoperability, or, where are these databases which can capture arbitrary data for arbitrary use cases, as well as enable cheap data flow and easy migration between them?[17]
Neither do I demand an answer here nor can I provide one. The only thing I can add is that there likely are certain things in Category Theory impinging on the question of modelling that I believe are probably underexplored. Worth looking into.
Footnotes
I am very proud I made it this far https://github.com/eighttrigrams/diffy, which includes back propagation (although for anyone learning this for real, this must be ridiculously unimpressive). What helped me getting this right, was some guy’s video and article which gave me expected input and output values from which I could construct test cases!
For anyone interested I suggest taking a look at https://bartoszmilewski.com/2014/10/28/category-theory-for-programmers-the-preface/. There exists an entire video series on this on YouTube as well.
Best introduction here probably: Concepts: Core Readings, edited by Eric Margolis and Stephen Laurence.
Susan Carey: The Origin of Concepts. [Oxford Series in Cognitive Development]
From the Critique of Pure Reason.
Which is basically Feynmann’s “What I cannot build. I do not understand.”
Again, in the Critique of Pure Reason.
And on that note, I want to point out a book which I found somewhat a curiousity: Diagrammatic Immanence: Category Theory and Philosophy by Rocco Gangle. I think diagrams reflect basic thinking structures. Which makes the premise of this book so interesting to me.
Brendan Fong, David I. Spivak: An Invitation to Applied Category Theory: Seven Sketches in Compositionality.
Edit: I actually found the video again that I was remembering there—after I finalised the article. It is this one, David Spivak: Categorical Databases - YouTube.
With, my guess, total loss indicating the impossibility of mapping.
If I’m not mistaken.
A database of “facts,” however encoded.
I have touched on the concept of local knowlege in an anthropological context in Best Way to Live - Local knowledge in Daniel Everett’s “Don’t Sleep, There are Snakes - Life and Language in the Amazonian Jungle”. Substack link.
In such systems, we have entities like Customer, LineItem, User, Price, Currency, or similar things.
Edit: Above I already mentioned the video David Spivak: Categorical Databases. This link transports you directly to where he mentions that “40% of IT budgets are spent on information integration.”—a fact which ever since stuck with me and surely informed this article, even if I didn’t have a conscious recollection of this anymore when I wrote it.