Draft: My Personal Exocortex Software System

by Jan Malakhovski, version 0.20.0, updated , published , created

This document is a draft! Major parts of this document are not yet written/edited/published, some of the already published parts may change in major ways or get reordered, some of the #fragment anchors may change, ... without most or even any of those changes being reflected in the Changelog

Having said that, the fact that this document is published in its current state means that it is already useful.

This notice will be removed when everything I publish everything I plan to write at the moment, at which point the news feed will say so.

Do you want to become an awesome cyborg who can easily track lots of medically, personally, and generally relevant variables, perform exploratory data analyses over them, refer back to previously experienced and related things at light speed, compute Bayesian probabilities for anything, apply the scientific method to everything, and use said abilities to optimize your life and knowledge-management? I do! This article discusses my adventures in designing, building, and growing my personal exocortex software system setup for the abovementioned purposes. It also describes how you can setup such a system for yourself.

Abstract

An exocortex (or exobrain) is a device that plays a role of “an external information processing system that augments the brain’s biological high-level cognitive processes”. In other words, it’s a device that does human intelligence amplification. At the moment of writing, this type of device exists only in the realms of science-fiction. However, if you only care about the software-defined information processing and intelligence augmentation features of such a device and not its neural brain-machine interface, then, with a bit of effort, you can have most of said features today.

Henceforth, a personal exocortex software system is defined as a software setup that does personal exocortex-like things, but using widely available conventional computing devices and operating systems. In other words, it’s an “extended memory” and “external brain” software system that

Should you have such a personal exocortex software system, you could then make a good approximation of a sci-fi-like exocortex device the moment a general wearable ubiquitous computer with enough compute, storage, and user-to-device IO bandwidth capacities appears.1

Why would you even want to use such a thing? Are there any potential dangers with using such a system? What is a good design for it? Which tools can you use to implement it? How can you make your setup private, secure, future-proof, efficient, and comfortable to use? This document aims to answer all of these and other relevant questions.

In other words, this document aims to be a comprehensive introduction, motivation, and DIY guide for wannabe tooling-, self-tracking-, and intelligence-enhanced cyborgs.

Glossary

Changelog

(Click me to see it.)

v0.20.0 - : stub → draft

v0.18.1 -

v0.18.0 -

v0.0.0 -

Table of Contents

Introduction

My history

Several people have been periodically reminding me that I’ve been promising to document my egocentric workflows for years now. To quickly sum up my experiences with these activities:

In other words, I built myself a personal exocortex software system, step-by-step, almost by accident, starting with unformatted plain-text files and simple grep, then switching to Emacs org-mode, then adding document archival and full-text desktop search, and then gradually migrating to my own tools that do a bunch more useful stuff. One of the following sections shall discuss my historiography of doing all these things and many implementation details of my system in detail.

My motivation

At first glance, for many of the abovementioned things, it might seem that the amount of return you could get by doing them with the time you would need to invest into them would make them counter-productive. But, I would argue that if you do it properly, this is not the case.

Health and productivity inferences

Many of my health-related self-tracking metrics I now pay careful attention to seemed completely absurd to me until I managed to isolate them out of my self-tracking data using exploratory data analyses, usually years later after I started tracking relevant variables, explicitly or not. (That is, as explicit time-dependent variables and functions built out of them, or implicitly, by converting time-tracking data into time-dependent variables, by inferring events from patterns of switches between time-tracked tasks and then turning those into time-dependent variables, etc.)

For example, for me, one of the metrics influencing my general health the most is the number of times I change clothes per day, apparently. The younger me from 2009 would have laughed in my face if I told him he needs to start tracking that without explaining the mechanism behind the observation. And this observation took me quite a while to figure out even after I had the data and saw the correlation. But knowing this now, I would have loved to know it in 2009 just as much.

Similarly, apparently, my productivity (defined as a reverse of procrastination) is most dependent on my Vitamin C intake and the amount of daylight I see, not my adherence to any particular time-management technique or the mystical “willpower”. This also was a revelation to me.

In other words, chances are, if you start self-tracking and self-experimenting, you’ll also probably discover highly personalized idiosyncratic life interventions that improve your life so greatly you’d wish you’d discovered them 15 years before.

One of the following sections shall discuss the general algorithm I employ in the above self-discovery workflow. Meanwhile, another of the following sections shall discuss the above and some other facts I discovered for myself about myself in much detail. The rest of the above ideas shall be discussed in more detail throughout this document.

Document archival and search

I found a system that automatically archives almost all of the web pages and other documents I experience and then allows me to search and quickly refer back to them without accessing the Internet to be absolutely invaluable. Unfortunately, on the Internet, things vanish completely all the time and change for the worse without older better versions being preserved sometime. As Gwern Branwen puts it:

[…] it’s always the apocalypse somewhere for someone […] none of that will survive on its own, [it] will be lost and forgotten, and if you don’t preserve it, probably no one else will either.

Sure, Wayback Machine and similar services are awesome, but they don’t archive everything, and they specifically don’t archive pages hidden behind CAPTCHAs, authentications, paywalls, HTTP POST requests, etc, thus making at least some amount of link rot absolutely inevitable and unrecoverable.

If nothing else, after I started using such a system, a situation where I remember a useful web page or other document on the Internet, want to look at it again, can’t find it among my bookmarks or using any of the web search engines, and so I eventually just give up, no longer happens to me. With my setup, unless I disable archival for a web page explicitly, everything always gets archived automatically. Therefore, if I can’t find something then I’m running the wrong search queries. Unlike with the Internet where things can simply vanish without a trace, with my archives, if I don’t give up, I will find it eventually.

This is especially true because I archive and full-text index not just most of the web pages I visit, but also all my emails, most of the files on my disk drives, a bunch of dictionaries, the whole of Wikipedia in all the languages I can read, a bunch of book and scientific article databases, fiction databases, food item databases, and many other things.

Therefore, in practice, I can usually find anything I search for almost immediately: my system searches through the documents I actually experienced first, through all those auxiliary databases second, falling back to doing Internet search last.

Thus, chances are, you can also benefit from such a setup.

My software setup for doing things like this and its possible alternatives known to me shall be discussed in later sections of this document.

Reminders with context and spaced-repetition

I found a system that combines a personal wiki with alarms/reminders to be invaluable for many of my everyday tasks. This might seem like a minor thing, but it really is not because such a system helps me to both

I found this to be incredibly useful for high-context many-steps-in-strict-order tasks that need to be repeated rarely enough that I forget most of the relevant details by the next time I need to do them again, especially if said tasks need to be executed very carefully and/or quickly when their time comes.

For example, when it nears the time when I should start thinking to renew my internet service subscription (which I do once a year because it’s much cheaper this way with my ISP), I no longer need to frantically search through the mess of my web browser bookmarks for the needed URL, then lookup my login and password there, and then try and remember in what order I need to press the buttons in their awful UI, all while praying I’m doing it right and not forgetting something. Instead, by using the above system:

That is, yes, I use my personal wiki to write tiny personal tutorials and checklists for my future self, not just for utilities and websites with awful UIs, but for a bunch of stuff, like my daily routines, important personal tasks, etc. If nothing else, after I started using such a system, I haven’t yet managed to completely screw up a release of a software project of mine, forget to pay for my utilities or DNS zone, or update my PGP keys.

This feature saves me a ton of time, effort, and stress.

With semi-automatic scheduling done via spaced repetition the above is also useful for making things stick in my brain. Many people use similar spaced repetition flashcard systems for drilling foreign and uncommon words. Some people also use such systems to drill their own document annotations and random facts. I do some of both too. Personally, however, I found this to be most useful for health-, learning-, and work-related checklists. Most of the time, I don’t actually need to drill their contents, just reminding myself that I have a specific solution for this type of problem from time to time is usually enough to save a lot of time potentially wasted in the future. In essence, this system periodically reminds me of things like:

Imagine having a personalized cookbook for living your life well that prevents you from backsliding from most efficient protocols to half-hearted ones and makes you more effective at everything as you age. It is exactly that.

My software setup for doing things like this and its possible alternatives known to me shall be discussed in later sections of this document.

Cruise control and data-driven nanny-ing

Since I do my time-tracking into the same personal wiki system, my task switcher tool can use the abovementioned checklists to simplify capture of sequences of tasks and help me keep on-track when I’m supposed to follow those sequences. Which is to say, when a checklist says that I should do something in a given order, my task switcher helps me do it in that order.

Moreover, when I’m not following a sequence of tasks defined by a checklist, my task switcher can also predict what I’m likely to do next via Bayesian reasoning and machine learning algorithms applied to my previous time-tracking logs. This simplifies my time-tracking capture significantly, since the most likely next tasks usually appear at the top of my task switcher menu.

More importantly, however, those predictions also allow me to generate pretty complex conditional reminder notifications, like “if my current time-tracked task has a history of making it harder to fall asleep (deep work, highly exciting entertainment, etc) and it’s getting close to bed time, start bothering me with progressively more annoying notifications that tell me I should stop doing that task as soon as possible and switch to something less exciting, otherwise I won’t be able to fall asleep on time”.

Yes, it’s a personal nanny system, but its suggestions are derived from my actual self-tracking data, not some arbitrary set of rules handed down from above by some unseen authority or reconstructed from many good-sounding but actually counter-productive pieces of advice by a large language model. Most of the time, those suggestions actually improve my daily routine and prevent me from backsliding. Also, in cases where I discover they aren’t useful, I can either fix their logic or disable them pretty easily.

My software setup for doing things like this and its possible alternatives known to me shall be discussed in later sections of this document.

Queries over personal wiki nodes and entities

Similarly, I found a system that implements a personal wiki with full-text search, backlink tracking, and related node/entity discovery, a.k.a. associative memex, to be incredibly helpful for knowledge management.

As Martin Edström puts it in org-node’s README:

My life can be divided into two periods “before org-roam” and “after org-roam”. I crossed a kind of gap once I got a good way to link [and view backlinks] between my notes. […] I used to lose track of things I had written, under some forgotten heading in a forgotten file in a forgotten directory. The org-roam method let me find and build on my own work, instead of recreating it all the time.

Except, my system is a bit more general than that as it treats all my archived documents and other entities indexed from supplementary databases as if they were part of my personal wiki, even if they have no personal wiki nodes associated with them yet. This produces a pretty nice knowledge management workflow because

Additionally, my system treats all variable references in my health-related logs as references to corresponding entities, so that, e.g., banana in my ATE 200g banana food diary record would be interpreted as a reference to the “banana” entity which lists its nutritional information, not just when computing things over those ATE records, but throughout.

Doing this allows me to abuse my personal wiki in many useful ways.

For example, say, I’m working on a note about something (e.g., this document) and then it suddenly hits me that I listened to a podcast episode related to the sentence I’m writing down at that moment and I should link to it. With my system

My system can also use all of the abovementioned data to implement associative memex on top of my personal wiki, document archive, and supplementary databases. That is, it can automatically discover potentially similar/related/relevant nodes/entities while I look at other nodes/entities, even when those things don’t link to each other, which can be done both on request or in background. The latter case is especially useful when one of those nodes/entities is the node/document I’m currently viewing or editing.

Which is to say, in the above example, it’s quite likely that my system would remind me about that related podcast episode as I’m editing the document in question even if I forgot about it, thus preventing most of the detrimental effects of cryptomnesia.

The above features make my personal wiki quite inter-linked, which, in turn, allows me to abuse my backlinks to generate lists of personal wiki nodes and/or documents of a certain type related to a given entity or subject. Like, “give me everything that ever mentions Bill Gates”, “generate me a list of all the books ever mentioned by that YouTuber”, or “get me a list of things I usually eat together with bananas”.

The above abilities are quite useful in isolation, however, I found them to be especially useful after my system learned to apply Bayesian reasoning to my personal wiki nodes, documents, and other entities, so that my tags and rankings could be suggested automatically when editing things and inferred automatically when querying for things. That is, my system can search not just for stuff like

but also for stuff like

which works even if I never experienced nor tagged/rated/scored any of them!

For example, if I assign high ‘awesomeness’ ratings to a bunch of documents referenced both by a certain podcast and by the set of documents linked from a certain web page (note a level of indirection), Bayesian reasoning can then automatically infer that everything else satisfying this criteria should probably be rated with high ‘awesomeness’ by default too. This, of course, can be applied to an arbitrary level of indirection, similarly to sports rating systems and Pagerank, which then allows me to use my system for ranked entity list generation, like

which can also be used as a recommendation system, since it will rank even the things I did not experience yet, assuming I have enough information about them in my databases.

But it’s even cooler than that because those Bayesian tags and ratings can be used in other advanced ways.

For example, I occasionally fact-check my recorded (para-)quotes, possibly years later after they were made and I recorded them, possibly using new information I discovered since then, and tag them as “true/false to the best of my knowledge” or with things like “this statement is true if statements recorded in those two related quotes are also true”. Which then allows me to apply Bayesian reasoning to them and to their entities+subject pairs. Which is to say, when a bunch of statements by an author misinform me on a certain subject matter, all other statements by the same author on the same subject get automatically marked as “likely false” by default in my system.

This is useful for still unexperienced documents because it solves most of Gell-Mann amnesia for me by allowing me to browse my to-experience list (to-read, to-watch, to-etc list) while taking all of the above things into account. Which is to say, my to-experience list is a dynamically generated personal wiki node produced by querying my system for a “list of all the yet unexperienced documents ever referenced in any of my personal wiki nodes, with all of those which will probably misinform me filtered out, sorted by most likely ‘awesomeness’ I would assign them after I experience them”. That is, yes, in essence, it’s an incredibly advanced personal recommendation system which supports arbitrarily complex user-editable recommendation criteria and can easily explain where its decisions come from.

However, the above is also useful for the already experienced things because when those truthfulness flags flip, my system can automatically produce me a list of documents/facts/(para-)quotes on which I should update my own in-brain state to keep my own self properly Bayesian. Which is to say, when I learn that some statement made in a (para-)quote I recorded before is false, and mark it as such in my system, my system can then tell me which of the other related things I probably still think to be true are now probably false and I should probably go and change my mind about them.

My software setup for doing things like this and its possible alternatives known to me shall be discussed in later sections of this document.

Takeaways

The above are not the only reasons why I can’t imagine myself stopping to use my personal exocortex software system, additional motivations shall be discussed in later sections of this document as they become relevant, but to sum all of them and the above up: integrated document archival and personal wiki system with full-text search, associative memex, typed time-dependent variables, data structures, Bayesian queries, and alarms/reminders is an incredible brain enhancement tool. Personally, I like to think of the effects of using such a system as being most similar to the effects of being literate: such a system makes many small seemingly inconsequential tasks very easy to execute with almost no brainpower being spent in the process, which, after a certain point, adds up to a qualitative increase in productivity. Then, after a while that productivity increase becomes normal and you completely forget about it until you suddenly loose access to those features, similarly to how traveling to a country that uses an unfamiliar writing system feels for literacy.

My assumptions

Now, before proceeding, ask yourself this:

What do I want to get out of those self-tracking- and knowledge-management-related things, what is my goal here?

Which is not to say that you can’t update your answer in the future or even while reading this document, but it’s an important question to consider.

As far as I can see, any answer to it will contain a mixture of the following elements:

So, my own answer to the above question is as follows:

I find these things to be fun. I like trying out new self-tracking-related things. I collect as much useful data about myself as I can as a sport. Which is especially true in cases when it’s relatively easy to do. I have a certain annoyance threshold there, but it’s almost certainly higher than it is for most other people.

I care about knowledge-management because I’m spending a large proportion of my life experiencing entities. Thus, most optimizations there produce immediately noticeable effects for me.

I’m all-in on baselines and tracking of changes to as much variables as I can.

I’m all-in on applying the scientific method to and self-experimenting over myself. But, only in ways I’m pretty sure are safe and sane.

I’m okay with nanny-ing only when it does not interfere with my self-experimentation and when the rules the nanny enforces are data-driven and I confirmed they actually improve things with self-experimentation.

However, importantly, I’m categorically unwilling to use tools that restrict the set of my future actions/choices. Which means that, generally speaking, I’m unwilling to use devices and software that violate my software freedoms (since violating those usually locks me into “walled gardens” that are hard to leave without also leaving my data there) and/or privacy (since violating it takes away the choice of not sharing my data in the future).

Many people I admire and/or care about do a lot of the stuff I won’t ever do (for the abovementioned freedom and privacy reasons), while not doing any of the stuff I think they should be doing instead. Some of them even mock me for doing some of these things sometimes. I don’t care. I like doing the things I’m doing. My measurements show me that I’m making things better by doing them. So, “live and let live”, “haters gonna hate”, and all that.

I care about the social signaling part only up to the point where the devices I carry around neither scare other people away from me nor make them curious enough to randomly question me about them. Which is to say, everything that looks weird must be easily done in private or hidden away.

This document assumes these preferences to be self-evident. I will occasionally mention things that violate some of them, but only in the context of on-boarding/bootstrapping into setups that can be made to satisfy them eventually. But the overall goal of this document is to criticize the state of the art from the above viewpoint and then describe the best setup that satisfies the above preferences (that I could think of, design, and then implement).

Meanwhile, if you, for example, weight social signaling first and foremost, then

I wear a fitness tracker to show others that I care about my health! I wear the most expensive one to show them I’m rich! I don’t even look at any of those stats ever! I care about privacy too, so I simply unpaired my tracker from my phone and uninstalled its data synchronization app.

is also a perfectly rational way to do it.

In other words, if your answer to the above question is sufficiently different from mine, the rest of this document will, at best, be of academic interest to you.

The state of the art

Now, I’m not the only person doing these things, though, from what I can see on the Internet, I’m probably one of the first people who took many of these things to their logical conclusions under the above assumptions. A bunch of people I know or know of are or had been self-tracking and archiving at least some of the same data as I do.

For instance, quite famously, Aleksandr Lyubishchev (in Russian) had been tracking his time usage almost continuously from 1916 to 1972, in 15 minute intervals, using only a clock and a typewriter, see there (in Russian)2. Many others wrote about their methodologies and tooling more recently, see Stephen Wolfram, Bernt Hansen, Gwern Branwen, ditto, karlicoss3, ditto, Martin Edström, ditto, and Ivan Medvedev.

In fact, in recent years some variants of this endeavor became popular enough to get “quantified self” search term, directories of documents dedicated to it, and a whole market of devices (like fitness trackers and smartwatches), smartphone apps, and web services that are supposed to help their users do it well.

Additionally, some other people do some exocortex-like things using their personal wikis, most notably:

My issues with the state of the art

So, when I was reminded of my promise to write this document once again, I thought I’d simply gather some links from that state of the art that I approve of, pepper them up with some of my experiences and comments, and go back to reworking my own tools for public consumption. “Surely,” I thought, “I don’t really need to explain how any of this is supposed to work, right?”

But then, while rereading my bookmarked articles on the subject (all of which can be found by following the links and directory references above deeply enough) and checking out any new stuff that seemed relevant that appeared since, I couldn’t help but notice that all self-tracking methodologies and device-based solutions I could see being discussed on the Internet — both the DIY ones and the ones marketed by manufacturers of consumer-grade devices, apps, and services — suffer from the following issues:

One of the following sections shall discuss these and other relevant issues in more detail.

Additionally, it appears to me that most people who develop and maintain personal wikis use them as simple textual knowledge databases instead of using them to their full potential. Which is to say, at the moment of writing, the “personal wiki crowd” and the “quantified self movement” don’t really intersect, even though a well-maintained personal wiki full of structured data, timestamps, and/or time-tracking data can be a very nice and easily quantifiable data source about your own self.

For example, even a very simple personal wiki system into which you dump your random timestamped ideas can be used for quantified “ideas per week” measurements of personal creativity, especially if you store or tag “good” and “bad” ideas separately. Add a bit of time-tracking, some regression analysis, and such system could probably already start producing useful self-improvement suggestions of the “if you do more of that, you’ll get more good ideas” kind.

Similarly, with the abovementioned wiki management of alarms/reminders, even in Emacs org-mode with its absurd number of features, it’s a rarely-mentioned extension (org-alert) and not a builtin, which makes most users unaware of these use cases. Meanwhile, the most interesting uses of such a system mentioned above are rather subtle and probably a bit too computationally expensive to be implemented in a low-performance language like Emacs LISP.

In other words, while re-researching the modern state of the art and re-reading my bookmarked articles on this subject, I discovered that

This document’s structure

The rest of this document is structured as follows.

This section is a work in progress.

Common pitfalls

This section discusses the most common ways people fail at collecting and analyzing their data and what you should be doing instead.

Expecting it to take very little time and effort right off the bat

Many people start self-tracking with unrealistic time and effort commitment expectations.

The unfortunate fact is, the first two weeks of any sufficiently new self-tracking activity are probably going to be torturous:

The above, of course, also applies to data analysis: expect that getting anything useful out of your collected data will be torturous for the first time too.

Educating yourself on other people’s experiences with these things, like you are doing reading this document, might help you smooth some things over, but it won’t magically resolve most of the abovementioned issues.

In other words, if you plan to self-track something, you never did it before, and you don’t consider writing things down obsessively and analyzing things with statistical tools to be fun in itself, you should go in expecting the first two weeks or so of the tracking process and then the first time you try analyzing your data to be absolutely torturous to do. Eventually, you will probably learn the minimal effective set of variables you need to track and how to do that tracking and subsequent analyses in a handful of minutes per day by automating most of these activities, but if you go in expecting it to take no time and effort from the very beginning, you are going to sour on this whole self-tracking thing very quickly.

Spending too much effort setting everything up right off the bat

Many people start their self-tracking activities with a lot of needless and, sometimes, counter-productive preparation. As noted above, you’ll know very little about what variables you would actually want to self-track using which methods at the beginning of your self-tracking activities. This is especially true in cases when you don’t yet know which behavioral modification methods work for you.

For example, for jogging activities, do you want to track simple time and distance? Or, maybe, steps taken? Maximum speed? Full tilt running only? Which methods do you use to force yourself to do it regularly? Which methods do you use to track your progress? How do you ensure they won’t demotivate you in times of negative progress? Which means that any time you spend browsing for shoes and outfits, habit tracking apps, geolocation-tracking apps, data analysis apps, etc, and then testing them all out before you do the actual jogging bit for a couple of weeks will probably go to waste.

The above also applies to other things discussed in this document. Like, you probably should not be spending a week of time setting up your ideal personal exocortex software system with all its bells and whistles if you don’t ever archive and annotate any documents. Start with something simple like annotations in SingleFile, if you are still doing it two weeks later, start working your way up. Similarly, don’t go setting up custom ripgrep or recoll filters for all you archived files if you don’t even grep regularly yet.

In other words, you should be applying efficacy optimization rules to your setup procedures too.

Also, as a rule of thumb, you should be prioritizing your efforts towards:

Collecting junk measurements

As far as I can see, most people doing the quantified self thing usually collect junk measurements for variables that are useless for learning new things about themselves.

While doing research on this, I actually became convinced that the “quantified self movement” is simply a marketing strategy designed to sell measurement devices, regardless of their actual usefulness for self-tracking and self-experimentation.

Many, if not most, of those measurement devices:

Most “quantified self movement” adherents happily track what the marketers of all those internet-of-(spying-on-you)-things devices want them tracking, but, personally, I found that the most useful devices for medical self-tracking are a photo camera and a tape measure combined with a text editor or a note-taking app.

This is why I prefer “self-tracking” and “self-measurement” terms over “quantified self” since, as the rest of this document is going to be repeatedly pointing out, most useful health-related measurements require no quantification use at all and most useful quantified self-measurements can’t be captured with any internet-of-(spying-on-you)-things devices known to me.

Now, to expand on the most common quantified self data collection issues mentioned above, let’s discuss why tracking many popular personal medical metrics using consumer-grade devices produces junk data.

Fitness trackers and Smartwatches

Fitness trackers and smartwatches that include similar functions suck at measuring everything they are marketed as being designed to measure. I feel like I’m getting hit over the head every time I hear otherwise technically competent people discuss those devices as if they measure anything useful for anyone but advertisement and health insurance companies.

A fitness tracker is essentially a combination of a digital thermometer (almost always a thermistor), a digital accelerometer, digital heart rate monitor (usually, an optical one, so, essentially, a discount version of pulse oximeter), and a GPS receiver.

Firstly, the GPS receiver part seems superfluous in the above list, doesn’t it? Why do you need that, when you have an accelerometer already? Shouldn’t the accelerometer be enough for step counting and other physical activity tracking?

Unfortunately, it’s not. Small digital accelerometers are pretty much useless for medical-grade measurements, which is what those devices would need for precise energy utilization calculations. Human body has a much wider acceleration range and more degrees of freedom than a small wrist-mounted device can possibly measure.

As a result, fitness trackers frequently fail even at leisure step counting. The wrist is simply the wrong place for these measurements, pedometers are supposed to be attached to a waist or an ankle. Adequate measurements for jogging and more vigorous activities via a wrist-mounted device are a fantasy. This is why, when outside, fitness trackers actually use their GPS receivers to approximate step count measurements.

Additionally, fitness trackers are incompatible with some fitness activities even in theory. For example, any activity that requires gloves that go over your wrists — like many strength training exercises or bag-punching — will interfere with their operation.

Also, simple acceleration measurements can’t track energy expenditures of most strength training exercises and cycling because they need to account for resistance, terrain, bike gear settings, ambient temperature, etc.

Secondly, if you ever played with an actual pulse oximeter or a similar heart rate monitoring device, you might have noticed that you can produce 5–10% variation in oxygen saturation and 50% or more measurement in heart rate by simply repositioning it, or sweating a little. 93% to 98% oxygen saturation range covers everything from “you should be in hospital!” to “perfectly healthy”.

Which is not to say that modern wrist-mounted heart-rate monitors can’t be accurate. They can be, at rest.

Trying to do exercises with such a device attached is a lost cause, I was never able to make any of the ones I tried to measure even the heart rate anywhere accurately when compared to the classic way of manual measurement by counting carotid artery beats for 6 seconds and multiplying by 10. They just can’t stick to your wrist and/or fingers tightly enough for those measurements to work properly while you shake them around doing sports.

Thus, in short, wrist-mounted fitness trackers actually suck at measuring fitness activities.

On the other hand, a wrist-mounted fitness tracker can track the following signals pretty well:

To sum up, it just so happens that fitness trackers and smartwatches suck at measuring everything except for signals that are most desirable for advertisement and health insurance purposes. These devices violate your privacy in rather creepy ways in return for almost no useful data at all, helping you to quantify yourself to your own detriment. Everything useful they are supposed to track can be tracked more precisely, in ways that keep your data private, with very little effort.

Additionally, overusing such tracking devices puts you under risk of Goodhart’s “law” becoming applicable to you.

As far as I can see, such devices can only really be used as a toy, a social signaling implement, and/or a nanny-ing tool, since most of these devices can remind you to do your exercises, go to bed on time, etc.

However, given that, objectively speaking, such devices track very little data useful for proper nanny-ing, the best of them are much less effective than the predictive nanny-ing method mentioned above.

Additionally, nanny-ing via simple criteria like “10000 steps per day” does not work all that well unless your schedule is very regular. Therefore, in my observations, most busy people who use devices with such nanny-ing functions eventually become annoyed with interruptions they introduce and/or get demotivated by their stats, and either disable those functions or stop using such devices entirely.

Also, the abovementioned privacy drawbacks make all of these uses rather counter-productive. But, even a very privacy-conscious open-source firmware/software open-hardware fitness-tracker-like device designed to be wrist-mounted will probably be pretty useless for the abovementioned reasons anyway.

On the other hand, note that some devices that have a separate chest strap for heart rate monitoring are usually pretty okay at measuring that and Tom’s Guide claims that Apple AirPods produce comparable measurements by measuring things from inside ears. If the latter claim can be replicated, then it’s kind of cool. Though, at the moment of writing, all smart earbuds require the use of smartphones that leave no privacy for their users, which makes such devices counter-productive too.

Generally speaking, I think that heart rate monitoring with a chest strap or equivalent is the only useful thing here. If you do cardio exercises regularly, then a device that buzzes when your heart rate goes above or below your cardio interval is useful. This can be done with regular manual measurements, but it’s annoying.

Sleep trackers

I fail to see how sleep tracking with a wristwatch-like device, like a fitness tracker, could produce any useful sleep quality data at all.

Personally, I can lie perfectly still for hours, thinking about things or meditating, without actually sleeping. Meanwhile, when the room is hot, I will migrate around the bed towards cooler places, without actually waking up.

Also, whether I change position while sleeping is actually most strongly correlated with a type of pillow I’m using. You can easily replicate this by taking a dense scarf, folding it a bunch of times, and then using that as a “pillow” while lying on your back on a hard mattress. Changing positions, except for minor rotations of the head side-to-side, while using this setup is uncomfortable. Thus, if you do it right, you won’t even be able to lie on your side properly, and changing positions while asleep would immediately wake you up. Therefore, after you learn to fall sleep while using such a “pillow” and get through the first couple of nights of uncomfortable sleep with it, your body will learn to stay still in that original flat position the whole night very quickly. Personally, I like this way of sleeping quite a bit, but you mileage may vary.

The point is, for me, movement while asleep is not at all correlated with sleep quality. What is correlated with sleep quality is fall-asleep time consistency, room and body temperatures, diet, Vitamin D levels, physical activity that day, stuffed sinuses, and sleep apnea. Most of these can’t be measured from a wrist and their effects are long-term, not immediate.

However, what sleep trackers can detect pretty well is the time when you go to bed, which is the main measurement that health insurance companies want to know here.

Also, note that some chronically insomniac people attempt to use sleep trackers to solve their issue, and get orthosomnia instead, thus making their insomnia worse.

Body mass (weight)

Body mass measurements via domestic floor weighting scales are highly dependent on positioning of both of your body on the scales and of the scales on the floor.

Firstly, even assuming your scales themselves are absolutely perfectly consistent, to get consistent measurements out of them you’ll need to:

If you got lazy with any of the above, your results will be rather approximate and inconsistent between measurements.

Medical scales solve the above issues by being large, bolted to the floor, and using counter-weights for calibration.

Secondly, even if your fat and muscle mass are completely static, your body mass will still vary throughout the day and throughout the year. Depending on your diet and the climate, 2–10% of your body mass could be in food and extra water content at any given point.

Therefore, to get consistent body mass measurements, you’ll need to weight yourself at the same time of day, with your urinary bladder and digestive system as close to the same state as possible. I.e, for instance, you could weight yourself every morning right after you visit the bathroom.

Even then, the results will probably be variable enough that 3-to-7-day running averages will be needed to make any sense of them. If you fail to account for this properly, you will probably make incorrect inferences from that data.

Though, internet-connected scales will happily report your weight to health insurance companies anyway.

In short, you should be measuring your body circumferences instead.

Body fat percentage

Body fat measurements via bioelectrical impedance (which is what domestic devices use) depends on a ton of factors:

Also, note that your body fat percentage measured via impedance will typically increase both when you get fatter and when you grow more muscle mass, which, in my opinion, makes impedance measurements completely useless.

An accurate measurement of body fat content can be made by analyzing an X-ray, but then, to track it, you will need to do full-body X-rays like once a week, which will give you cancer.

So, again, you should be measuring your body circumferences instead.

Blood glucose, when not logging Vitamin C intake

A little known fact is that electronic blood glucose meters (glucometers) and most lab tests measuring blood glucose actually measure linear functions of blood glucose levels and levels of various ascorbates (Vitamin C variants like ascorbic acid, sodium ascorbate, dehydroascorbic acid, etc, there’s a bunch of them), which are all molecules very similar to glucose, and all such molecules contribute to the resulting measurement with different weights depending on precise measurement method employed.

That is, if you are not controlling for Vitamin C intake and usage, absolute values of your blood glucose measurements are most likely completely useless.

Personally, after I started mega-dosing with Vitamin C, I got really confused the first time I was measuring blood glucose impact of a new food item. I was like, “Wait, how many moles per liter?! Did I develop diabetes all of a sudden?”, so I checked again, and got the same result. So I then looked at my self-tracking log and immediately noticed that the only possible relevant change in my routines was me taking a mega-dose of Vitamin C ~3 hours before. After which I looked it up and discovered that glucometers (and most other glucose measurement methods) indeed do not play well with Vitamin C.

My self-experiments have shown than with high-enough mega-dose of Vitamin C I can actually make my glucometer produce absurdly high measurements. It then takes me at least 5 days of no Vitamin C supplementation to get back to my baseline. Also, for me, going from near-zero Vitamin C intake to mega-intake actually decreases my glucometer measurements before making them increase out of control.

I hypothesize that what happens there is that glucose and Vitamin C compete for blood transports (which is a well-known effect), my glucometer measures Vitamin C as if it’s glucose (which is also a known effect), but its sensor captures Vitamin C molecules with smaller probabilities. Therefore, after a period with low Vitamin C intake, the very first high Vitamin C dose drops my glucometer-measured “blood glucose” level below my normal baseline, but then subsequent doses raise that value back up to and then way over it. This mechanics, if true, explains a lot of confusing statements about the effects of Vitamin C intake in diabetics I saw while doing my research into this, as some studies and individuals claim that their measured levels fall, while others claim that their levels increase.

As a result, because I found these variables to be important for my analyses, as shall be noted below, in my tooling I actually wrote some code that models my Vitamin C biochemistry to approximate my absolute blood glucose and Vitamin C values from my raw glucometer measurements combined with my food and supplementation log records.

But, the point is, if you do not take dietary and any supplemental Vitamin C you intake into account, your glucometer measurements are probably pretty imprecise and inconsistent.

Though, if you only want to measure blood glucose impact of a food item, then only the difference between measurements matters. I.e., you only need to ensure that the measurements are consistent, the absolute values do not really matter. Blood concentration of Vitamin C stabilizes in ~3–6 hours after intake (depending on the dose and usage; when ill, my organism hoovers up all my Vitamin C intake before an hour passes), so, assuming you were supplementing consistently the previous few days, any measurements taken after that will be consistent enough.

Also, while researching this I found that, apparently, some doctors don’t recommend diabetics take Vitamin C exactly for the reason that it screws up blood glucose measurements. Which is crazy, because one of the main functions of Vitamin C is that it gets used to produce collagen, which is the main thing maintaining blood vessel health, which is what diabetes damages the most. So, to preserve the consistency of blood glucose measurements, some doctors recommend their patients to stop taking the actual thing that is keeping them alive in the long-term. What the actual f*ck!?

Failing to maintain privacy while collecting data

As the following sections shall demonstrate, the kinds of data one needs to track to get useful results ranges from “it would be a bit embarrassing if this data was ever stolen from me” to “please destroy all my data drives and/or encryption keys upon my death”. You would not want to give that much private information to a third party. Even to your doctor you would ever only give a tiny subset of it. Which means that the data you collect needs to be handled more carefully than your medical records.

Unfortunately, most modern computing devices and software are designed to deny their users that kind of control.

Failing to maintain privacy with web services

For web services, making a multi-user web service for self-tracking satisfying the above privacy concerns requires a level of care similar to that of Tahoe-LAFS and Tutanota. I.e., it needs to be built on top of end-to-end encrypted storage satisfying the least-authority principle.

Systems like that are quite complex. But I also fail to see any benefits to the multi-user feature that can not be achieved by the users simply selectively sharing some of their tracking (manually or semi-automatically). It’s not like people can collaborate on this, you friends can’t track your metrics for you.

Therefore, I fail to see how using a (non-local) web service for this makes any sense at all.

Failing to maintain privacy on smartphones and with their apps
  1. Cloud backups that steal your data

    Without installing alternative firmware onto your Android smartphone, how do you prevent your private data from being helpfully “backed up” to the cloud without it being properly encrypted in a way that prevents the cloud from reading that data? I.e., how do you prevent it from being stolen by Google, Apple, Samsung, Microsoft, etc?

    You need to tweak a bunch of enabled-by-default settings, which is annoying. Having those settings there should also imply they do what they say they do, which they frequently don’t. And then regulators slap Big Tech companies with fines less than 0.1% of the profits made by stealing that data, so they have no incentive to stop doing this shite.

    And then, to add insult to injury, OS updates also periodically silently reset all those settings to their default values.

    That is, in practice, you’ll probably fail to prevent Google, Apple, Samsung, Microsoft, etc from stealing at least some of your data unless you are running a privacy-conscious Android fork or just straight up GNU/Linux on your mobile device.

  2. Evil spying apps

    Similarly, without running an alternative firmware, how do you prevent a random app that has “media access” and/or “document access” permissions from stealing all your private photos and/or other things?

    You can’t.

    Many popular apps do this all the time by scanning all files they have access to — slowly, bit by bit, to make their activity less noticeable — and then uploading anything interesting their servers don’t have a copy of yet. In some countries, some of those apps are even required to do this by law. Supposedly, to protect the children or some such bullshit.

    A good smartphone OS designed to be under its user’s control would instead isolate all apps by default — i.e., each app would get its own “media” folder — and allow its user to partially merge those sandboxes so that one could, e.g., allow Syncthing file synchronization app access to both Open Camera photo camera app’s photo directory and a VLC media player app’s music directory, but prevent Open Camera and VLC from touching each other’s files.

    Unfortunately, such a thing does not exist. There are ways to approximate it, though.

    On stock Android, Shelter can isolate evil apps away from good apps using “Work Profile” stock Android feature. But, unfortunately, it can’t isolate a single app from all the rest, only split your apps into two sets isolated from each other.

    On GrapheneOS, “Improved User Profiles” feature can split your apps into arbitrary number of isolated sets.

    Unfortunately, both Shelter and GrapheneOS’s Improved User Profiles make backups for those isolated apps much harder.

  3. Evil keyboard apps

    Additionally, note that most Android keyboard apps are essentially keyloggers as they send everything you type to their developers.

    All of your passwords and chat messages will be analyzed to sell you stuff.

  4. Evil TTS apps

    Similarly, many third-party text-to-speech (TTS) apps also send everything you give them to their developers.

    All of your e-books and by-turn directions will be analyzed to sell you stuff.

  5. Evil system apps

    Similarly, on most Android firmwares coming pre-installed on Android devices, system apps also constantly spy on you.

    For example:

    • you app usage statistics (how much time you spend using which apps) probably get regularly sent to your device’s manufacturer;
    • every time you share something using a “Share” button, your system’s share widget also probably quietly shares what you intend to share to another app with the manufacturer.
  6. Takeaways

    Therefore, I find it unlikely that a sufficiently private self-tracking workflow can be achieved on a smartphone, unless it runs GrapheneOS or similar alternative firmware. Or, unless you first commit quite a bit of effort to specifically resolve some of the abovementioned issues, and even then, it’s not a 100% guarantee.

    Though, the minimal effective self-tracking protocol can be implemented in a private-enough way on most devices anyway.

Failing to maintain privacy on Windows and with its apps
  1. OS-level advertisements

    Advertisements-wise, modern Windows is worse than most consumer smartphone OSes known to me. Advertisements in the “Start” menu are absurd, okay. I don’t know any other OS that shows you ads every time you want to launch an app.

  2. Telemetry you can’t really disable

    Most Windows 10+ telemetry can only be disabled via Windows Registry. And all that telemetry gets silently re-enabled on OS updates.

  3. Windows Recall

    Windows Recall is a feature that is essentially equivalent to a software package North Korean government requires to be pre-installed on all computing devices there.

  4. Cloud backups that steal your data

    Windows 11 and later does its best to trick you into creating a Microsoft Account, saving your BitLocker encryption keys into there, and then saving your files to OneDrive by default, so that Microsoft could scan those files at their leisure, like Google and Apple do.

    Sure, they proudly claim that unlike with Android and iOS, unencrypted backups to OneDrive are not the default (yet). So, all your personal files they have not yet tricked you into saving onto OneDrive and your Windows Recall databases are completely safe. Until they change the default value of that setting to a more helpful value on update, opting you in by default.

    Apple does this and keeps the files even after you manually delete them, see this and that. Google does this and also probably keeps everything, though they are much better at maintaining the illusion they do not. I’m actually surprised Microsoft does not (yet). Maybe most Windows 11+ users save most things to OneDrive by default and don’t even notice anyway.

  5. Takeaways

    Therefore, at the moment, on a desktop or laptop computer, you will probably have to run Qubes OS, or fiddle with program isolation in your favorite GNU/Linux distro.

Failing to future-proof collected data

The vast majority of modern consumer-grade software solutions fail “Will I be able to access this data 100 years into the future?” test. But even without that, they frequently make access to data sub-optimal.

Failing to future-proof web service data

Firstly, how do you automatically download web service take-outs with a cron job to ensure you always have the latest copy?

Most of the time, you can’t.

Personally, I have lost quite a bit of data and effort to such services in my youth. The case that pissed me off the most was Spring.me/Formspring which was a popular “Ask Me Anything” service I liked, but it run out of money, was sold to a dating service, which simply threw out all user data immediately, not allowing anyone to backup anything, converted all user profiles into dating profiles without any opt-in, and started harassing me with my “dating options”.

In fact, generally speaking, most web services die before reaching the age of 10. Those that do not die eventually pivot away from all data- and computationally-expensive features and/or start inflicting Cybercrud on you while Enshittifying themselves, see a sibling note on the subject.

You can ameliorate this issue somewhat by downloading data take-outs from those services regularly and computing all those inferences yourself with code written with the help of HPI or something similar. But then, why are you even using those web services in the first place? In the long-term, what benefits are they providing to you, exactly? Violations of your privacy in return for you implementing parsers for their take-outs and re-implementing all their useful features yourself? Forcing you to use HPI and similar software as recovery tools? In the long run, it takes less work to do the whole thing from scratch yourself!

Failing to future-proof data on smartphones and with their apps

From a law standpoint, smartphones and their apps are one of the easiest places where Cybercrud and Enshittification can be introduced. See the explanation of the mechanics there. Most modern-day smartphones come with

In other words, those devices are essentially designed with anti-circumvention and planned obsolescence features in mind.

Additionally, generally speaking, mobile software longevity is atrocious. Mobile computing historically had very high levels of churn, see defunct mobile OSes like Palm OS, Windows CE, Symbian, Windows Mobile, and Windows Phone. Similarly, many older apps no longer work with newer versions of Android and iOS and most newer apps are impossible to run on older devices. For the above list of OSes, I’m only aware of decent system-level emulation tools for Android by the way of Waydroid and the way it’s designed does not make me feel confident it will stay relevant 20 years into the future either.

As a result:

“Can I safely browse the Internet on a laptop that is 20 years old?”

“Probably, assuming it’s not made by Apple who love gluing batteries in. Replace the battery, install a GNU/Linux distro for older computers, install a modern lightweight browser onto it, ta-da!”

“Can I safely use a smartphone that is 10 years old?”

“Probably not. Most of them now glue their batteries in like iPhones do, so it probably won’t even turn on disconnected from its charger. All software on it will be dangerously outdated. You won’t be able to make it run newer apps.”

Moreover, while Android was relatively more future-proof than the other mobile OSes in the past, with Google’s recent changes to Android this is no longer the case, most notably:

In short, smartphone apps are not a good solution for long-term self-tracking data collection. (Though, obviously, using them for capture of self-tracking data that you then export out to a desktop/laptop PC is fine if you can do it in a private-enough way.)

Failing to future-proof data using Microsoft’s products, including Windows

Microsoft’s is notable for promising indefinite compatibility for its file formats, which they then never deliver. Notably, none of RTF, DOC, DOCX, and other idiosyncratic file formats that used by WordPad and MS Office have complete up-to-date public specifications, regardless of what anyone claims. Those programs regularly produce files that don’t conform to Microsoft’s own documentation. Also, files created by old versions of those programs regularly fail to be interpreted properly by newer versions of Microsoft’s own products.

In short, you probably shouldn’t be using these file formats for anything long-term. If you have an RTF/DOC/DOCX file you want to be able to read 20 years into the future, (pseudo-)print it into a PDF. You can then store both files near each other in case you ever need the original DOC/DOCX again.

Minimal future-proofing requirements

In my opinion, automatic local backups into file formats that are likely to survive the next 100 years should be the absolute minimum requirement for self-tracking tools. Therefore, at the moment, you will probably have to use desktop software to satisfy this requirement. You might be able to get away with religiously backing up your data from your mobile apps, taking regular web service take-outs, and then taking care to ensure that data is stored in a future-proof way. But it means that you will probably lose at least some of your data eventually anyway.

Personally, I’m betting on the following file formats surviving the next 100 years:

Therefore, you should preferably be using tools that produce records in abovementioned future-proof file formats. Simpler formats are usually better than more complex ones. That is, when possible, you should always aim at recording things into plain-text instead of HTML and PDF, and similarly for others.

Additionally, a sibling note has a list of strategies and rules of thumb relevant for future-proofing of products and services I use myself. Though, that article is best read linearly from top to bottom, otherwise the reasoning behind some of those strategies and rules will probably appear rather mysterious.

Obsessing over pretty pictures and summaries instead of doing proper data analyses

Many quantified self adherents appear to be obsessed with drawing pretty pictures/graphs of “raw values over time” and “summary pie chart” kinds. Personally, I found such graphs of nearly-raw data to be either completely useless or even demotivating. So, instead of drawing those, you should be drawing “changes over time” graphs and doing exploratory data analyses instead.

For example, the English-language quantified self classic “The Personal Analytics of My Life” by Stephen Wolfram (2012) has lots of graphs of the “unit of a thing per day” variety. The Russian-language equivalent classic “A Kind of Voluntary Forced Labor” by Aleksandr Lyubishchev (1976, in Russian) is quite similar, but presents the time Lyubishchev spent doing various things in summary tables instead.

The me of 2014 was fascinated by those articles. The me of 2025 can’t help but ask “What new information about yourself can you learn by looking at all those graphs and tables?” Like, for instance, what new things could the graph of “raw numbers of outgoing e-mails per day” tell you?

When GMail analyzes your email usage, even by looking at outgoing message headers only, they could infer all kinds of useful stuff: your time zone, your sleep schedule (for some people, probably more precisely than their sleep tracker can), your work time, your interests and hobbies, your employment status, the number of projects you are involved in, your place on the corporate ladder, you likely remuneration, etc, which are all very salable fact useful for advertisements, but none of these facts should be new information to you.

Personally, in the past, when I looked at my “units of work per day” graphs and tables, with high probability, one of the two things happened:

A similar thing usually happened in instances when I looked at my raw body measurements. It was either

which then had the obvious effects.

Thus, in my experience, if you want to look at some graphs or tables, almost without exception, you should be looking at “changes over time” kind, not the “values over time” kind.

For instance, if you only ever track your waist circumference, and don’t track enough data for any other analysis, you should be watching changes to circumference’s weekly running average (preferably, as relative delta in percents) and read the results as “Am I doing the right thing to improve it lately?” and immediately backtrack recent changes to your diet if it starts going the wrong way.

Also, note that the difference between “waist circumference over time” and “changes to waist circumference running average over time” might seem rather minor but, personally, I never managed to demotivate myself by looking at the later graphs/tables, especially when looking at wide time ranges, like five years or more.

“But couldn’t I simply desensitize myself to the raw value graphs/tables instead?” you might ask. Probably. But should you even try, when you can trivially work around it all?

Meanwhile, in case you do track enough variables for useful fact inferences, you should be performing exploratory data analyses over your collected data instead of plotting simple graphs and generating simple tables.

For instance, Steven Wolfram shows he tracked a ton of different metrics about himself in his article. Assuming his “outgoing emails per day” metric was a good measure of his productivity — that is, if most days he wanted to write more emails than he actually managed to compose that day, i.e., if he was bottlenecked by email writing, which is probably false, but let’s assume it’s true for a second — then what could have been informative to look at is a set of graphs relating his email-words-per-day metric to fall-asleep time consistency, total sleep time, physical exercises, intakes of medicines, carbohydrates, proteins, fats, sodium, vitamins, etc.

Better yet, he could have just generated a linear regression coefficients table between his email-words-per-day metric and everything else he managed to track (or use a more expensive analysis method, but linear regression is cheap and is usually a good start), studied it carefully, tried explaining any surprises there, and then self-experimented by modifying his daily actions to see if the interventions this analysis suggested would have improved his productivity.

Unfortunately, this obsession with pretty pictures to the detriment of users is quite common. In fact, at the moment of writing of this article no consumer-grade software/apps, web services, or devices known to me do any of the abovementioned things properly.

All of them draw those demotivating graphs, none of them known to me do anything actually useful to you with the data they help you collect.

Some of them do those useful inferences about you in background and then sell those inferences about you to advertisement and health insurance companies without giving you access to them, though.

Ignoring the effects of Goodhart’s “law”

A rather famous statistical observation, commonly known as Goodhart’s “law”4, states that

Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.

or, as it is commonly paraphrased to laypersons:

When a measure becomes a target, it ceases to be a good measure.

This statement can be formalized a bit in a way that also explains why it usually works in practice:

If you have a control loop (measure outputs → analyze them → modify control inputs → repeat) with two correlated measurements “A” and “B”, then trying to control “A” by controlling “B” probably won’t produce the desired effect, instead, it will probably simply break the correlation.

The point of bringing this up in the context of self-tracking and self-experimentation is that Goodhart’s “law” implies that when you try to self-optimize, you should select your optimization criteria very carefully.

For example, I found that trying to maximize

all make me use vastly different strategies and daily schedules.

For example, “work items” is usually the easiest to count. But when I watch that metric too carefully, I start spending inordinate amounts of time on Getting Things Done bureaucracy by turning every little thing into a to-do item I could then mark as DONE later or immediately. “After all, I did actually do this, so my score should reflect it!” Which is not to say that Getting Things Done is useless, it’s a helpful planning technique when you are stuck. But using it too much makes me start creating lots of to-do items while achieving little to no forward progress.

Goodhart’s “law” is also the reason why I hate GitHub’s “№ contributions in the last year” thingy where it draws a year of day-boxes and colors days with different colors depending on the number of commits in master branches in all of your repositories on that day. It makes some stupid git workflow strategies vastly preferable to others if you want to make that page widget look impressive. Like, it penalizes feature branches and experimentation work using them, but it rewards splitting things into as many little pieces as possible for no rational reasons.

Or, more generally, Goodhart’s “law” is also the reason why most simple summaries are actually counter-productive.

Meanwhile, applying the above to fitness tracking implies that optimizing your daily life by using a fitness tracker or a similar device as your only source of tracking data puts you under risk of making you prefer (consciously and/or subconsciously) doing things your device can measure well, instead of doing things that are actually good for you. Which is another reason I think those devices are harmful.

Working around Goodhart’s “law”

Remember, what you want to track is variables that would make your future exploratory data analyses useful:

Ignoring the effects of the regression to the mean

Consider the following scenario. Something starts hurting a little, it isn’t that bad, so you ignore it. It gets worse after a while and you decide to go to a doctor after all. The doctor prescribes you a medicine and/or a treatment. You take it and/or perform it for a while and, eventually, your issue vanishes. You thank modern medicine for good service and forget about it. (Unless and until the issue reappears again.)

Interestingly, statistically speaking, for most medical conditions, most, if not all, of the same result could have been achieved by doing absolutely nothing for an equivalent amount of time. That is, people usually decide to visit a doctor when their symptoms are at their peak, meaning that, most likely, most of the improvement a prescribed medical treatment appears to provide comes not from the treatment itself but from regression to the mean — which is a statistical observation that says that a time-dependent random variable the current value of which is close to its extreme is likely to start moving towards its mean as time moves forward.

In the context of medicine, most of this effect comes from the immune system and other builtin self-repair mechanisms of the human body. Or, as Voltaire put it:

The art of medicine consists of amusing the patient while the nature cures the disease.

This, of course, does not imply that refusing to go to a doctor is the right thing to do in situations similar to the one described above. “Most likely” does not mean “always” and “most of the result” does not mean “all of the result”, especially since many treatments are much more effective now than they were in Voltaire’s times, and a diagnosis is something that should be performed in any case. But it is something to be aware of when measuring and considering claimed effectiveness of medical treatments.

On the other hand, in cases where no diagnosis can be made and for diagnoses where contemporary medicine has no good ideas about what to do, the above implies that doing nothing is usually the right choice. Which explains how “faith healers” can sometimes be very effective: if they have enough authority to shoo away dispensers of the ineffective and/or harmful medical treatments, then “let’s pray for their health” therapy is exactly the “do nothing” treatment.

For instance, notoriously, Grigori Rasputin was quite effective at “healing” tsesarevich Alexei, including remotely, from half-a-Russia away, by employing his mystic ways of… getting all doctors away from him. Which, I imagine, worked especially well with the doctors shoving blood-thinning aspirin into already hemophiliac Alexei to help with his pains. Then, I would imagine, things would get randomly worse again for Alexei, which would allow normal doctors muscle their way back in again and “win” with regression to the mean now on their side, but then things would get worse again, especially with aspirin making things much worse much quicker, and Rasputin would muscle his way back in and “win” with regression to the mean on his side now, but then…

In short, poor tsesarevich Alexei.

I imagine many people in modern times have similar cyclical experiences with mainstream and alternative medical practices.

Unfortunately, even though the above observations are rather useful to know, they get acknowledged in popular discussions of medicine… essentially never, probably because real healthcare effectiveness numbers with regression to the mean effect subtracted out are rather embarrassing. However, in most civilized countries, with some exceptions of medical practices that are legalized medical malpractices, they are usually higher than zero, which is more than can be said about many other things humans do.

It’s even a more rarely recognized fact that regression to the mean effect applies rather generally to many aspects of real life. Once you properly internalize it, however, it becomes very hard to stop noticing it. It also has several important corollaries.

All of which we shall now discuss.

In the context of A-B-testing, regression to the mean implies that when measuring effectiveness of some proposed interventions/therapies/changes/etc, “no interventions/therapies/changes/etc applied” must be used as the control group. To it stated like this, most people will probably say “Duh! Obviously!”. However, it’s easy to say “Duh!” when you only have one intervention to test, but the above implies that if you have “no interventions” baseline “A” and two (or more) proposed interventions “B” and “C”, then you should be testing “A” versus “B” and “A” versus “C”, not “B” versus “C”, otherwise your effectiveness scores will be meaningless because of the regression to the mean. (Though, with big enough sample size you could still correctly recognize which of “B” or “C” is better.)

Unfortunately, medical trials frequently violate this rule and test “intervention” versus “assumed placebo” instead of “intervention” versus “no intervention”. That is, clinical trials usually test

As a result, effectiveness numbers for most clinical therapies are highly overstated and many “clinically tested” drugs, vaccines, etc are actually completely ineffective. (Note that FDA and similar agencies mostly test for drug safety, not effectiveness. So, even when they are not wrong about safety, as shall be noted below they frequently are, their approval means even less for effectiveness.) See “Placebo effects are weak: regression to the mean is the main reason ineffective treatments appear to work” by David Colquhoun (2015) and references there for more discussion on this subject.

Moreover, regression to the mean implies that to properly measure effectiveness of a set of interventions, you need to compare the initial state with no interventions at all against all possible subsets of that set of interventions or, if that is unfeasible, the initial state against the state with all interventions all at once, but not sequential step-by-step interventions compared against each other! This, too, might seem rather obvious when stated this way, but this gets done properly pretty rarely in most fields, not just in medical trials.

For example, in the context of software development, to measure actual progress over some software feature’s performance, at the very least, you should be comparing the earliest version of that software that has an equivalent feature against the latest version, not just its incremental versions against each other. Better yet, you should be comparing that early version with all new applicable optimizations against the latest version. After all, this is where the regression to the mean effect comes from here: something gets worse, something, possibly unrelated, gets optimized in return to make program’s performance better again. Unfeasibly better yet, you should be comparing that early version against all subsets of applicable changes.

If you think about it, the above observations then have some fun implications for most fields that employ (or should employ) A-B-testing, explicitly or not. Consider, for example, the above in the context of real-life management, be it normal business management, evidence-based law-making, or elections. After you start noticing those effects, you won’t be able to unsee them.

“This was very interesting. But get back to the subject of this document already!”

Alright, let’s discuss some consequences of the regression to the mean relevant to the subject of this document and what you can do about them.

Failing to confirm your findings with proper self-experiments

Imagine a situation where you want to optimize some personal metric “X”. So, you collect more than enough data, analyze it all properly, and your findings say that you need to do intervention “A” to improve your “X”. Do you simply go do “A” from now on?

No! Correlation does not imply causation! If you simply start applying all your suggested interventions, you will fall into performing ineffective idiosyncratic rituals until something drastic knocks you out of them. It’s surprising how many people fall for this and become “superstitious pigeons”.

What you should do instead is perform a proper intentional self-experiment over your intervention “A” to verify that the correlation you found is an actual causal effect. In other words, after you discover your potential intervention “A”, you should continue your self-tracking of all the relevant variables and then self-experiment using single-subject research methods.

The standard way to do this is to first do “A” for a while, then do “B” for a while, then repeat that sequence one or more times. The result is what is usually called “A-B-A-B” or “A-B-(A-B)+” self-experiment. Note that, for the abovementioned reasons, in most cases, your intervention “B” should be “do nothing”. Which is to say, you should try doing your intervention “A” for a while, see if it affects your metric “X”, then stop doing “A” for a while, see if metric “X” changes, if it does, then try “A” again, then stop it again, and then repeat until your are confident enough in your discovery or you falsify it.

In most simple cases you can actually perform this analysis manually by taking a calendar, marking days with “did the intervention” and “observed the result” tick-marks, and then eyeballing your causations.

Failing to implement the changes your findings suggest

Many people start tracking things not because they want that data for future reference, but with an intention of changing something about themselves, that is, for behavioral modification. However, many of those people expect their analyses to pat them on the back and say they are doing everything right or suggest interventions they are already aware of, while, ideally, suggesting small additional changes that improve things greatly.

This is not how it works in real life. Usually, the greater the possible improvement, the greater the required change. Which is to say, if you want to actually modify your behaviour in the long term, expect needing to commit extraordinary effort and/or to start using radical/idiosyncratic/weird methods to achieve your goals.

For example, say you never self-tracked anything ever but want to start tracking your jogging time and distance because you want to be able to run 8 kilometers (5 miles) in an hour. Which of the following outcomes is more likely?

For another common example, limited-term dieting is known to be ineffective or even counter-productive in the long term, see “Medicare’s search for effective obesity treatments: diets are not the answer” by Traci Mann et al (2007).

Therefore, if you plan to self-track for behavioral modification, you should figure out which behavioral modification methods work for you, use those methods to force yourself into making the required changes, and then stick with both. In other words, for the above examples:

Note that different behavioral modification methods work differently for different people. For some people, simple reminders with pre-set daily alarms suffice. (Jealous noises.) For others, calendars with colored boxes or ticks for various tasks work really well. (Jealous noises intensify.) For yet others, various mental tricks, like considering a task completed only when it’s done two days in a row, and then considering it incomplete the next day again work. (Like, seriously. What? How?)

For me:

So, personally, I had to get really creative with data-driven nanny reminders that display their notifications in creative non-repeating ways, constantly remind me why I want to do the things they remind me of, and do not relent until I do it.

Most people probably don’t need to go to such lengths. But, the point is, you will need to experiment to figure out what works for you.

In my experience, almost every time I self-tracked and then analyzed something important, my analysis suggested a radical change I was completely unprepared and/or unwilling to implement at first.

  1. Dealing with contradictions with field consensus

    The above can also get rather worse too. Imagine a situation where you want to optimize some personal health-related metric “X”. So, you collect more than enough data, analyze it all properly, find a possibly-beneficial intervention “A”, confirm it with a properly designed self-experiment… but then you discover that your “A” is exactly the reverse of what mainstream medicine says you should be doing. Do you actually do it? This is exactly what happened multiple times to me.

    It actually took me some time to figure out that the most effective way to proceed in this situation is to immediately stop wasting time trying to consult supposed experts and search for second opinions — none of those people know your exact situation! — and go straight to the source by applying the following algorithm:

    • Figure out the sources of those mainstream views, that is, get to the actual scientific references.

      You’d be surprised how often they cite nothing at all.

    • Lookup those references — on PubMed or publisher’s web site if those articles are open-access, on Sci-Hub and/or LibGen if not — to disambiguate the folk Internet interpretation of those mainstream claims of “doing ‘A’ is good, but doing ‘B’ is bad” into concrete “doing ‘A’ produces desirable effect ‘D’, but doing ‘B’ produced undesirable effect ‘U’”.

      Also, you’d be surprised how often you open the actual article, and it says exactly the reverse of what the folk interpretation says.

    • Then, do a bit more research to figure out if “D” is actually something desirable.

      You’d be surprised how often there’s disagreement of what even constitutes “good” in medicine.

    • If their argument for “D” is convincing, simply measure the variables they measure in your own self-experiments too.

      In most cases this is rather trivial to do since the effects are apparent. In others, you will have to take some blood tests.

    As a result of the above, you will either:

    • come to a conclusion that the folk interpretation is simply wrong;

      and so, you should ignore it;

    • come to a conclusion that “D” is not even desirable;

      in which case, you can also ignore the mainstream advice;

    • discover that the intervention your findings suggest also produces “D”;

      in which case, since you will achieve the same ultimate result, you can ignore the mainstream advice too;

    • discover an actual issue with your approach;

      in which case mainstream advice will be a good hint what you should be self-experimenting on next.

    If you are interested, a sibling note discussing nutrition has a bunch of examples where I start with a well-known mainstream nutritional advice, drill into the literature, and end with one of the first three outcomes above for every single mainstream claim I investigate.

Failing to collect enough data by not writing important things down and/or giving up too early

Note that most people perform implicit self-experiments, which are sloppy self-experiments where you do everything without writing anything down, all the time.

For example, when you notice that some foods make you feel better than others and start consuming more of them, you are actually self-experimenting. Say, for instance, you accidentally discovered that eating bananas appears to improve your mood. Then, it’s likely you’ve actually spent quite some time discovering the precise protocol details and the dose that provides the best effect — e.g., “less than 2 bananas per day are ineffective, but more than 4 bananas per day and I start getting fatter, more than 8 bananas per day can actually make me feel sick, but if I freeze and thaw them first, then up to 6 bananas per day produce zero ill effects” — without thinking about it explicitly, simply by trial and error, purely eyeballing it all, never noting any of those numbers explicitly. Food cravings are, essentially, implicit inferences made from such implicit self-experiments.

Similarly, when a doctor gives you a prescription and you simply take/do what it says, technically, you are also self-experimenting, since you are, at the very least, probably varying time intervals between the doses, which can produce easily noticeable/measurable differences. That is to say, if you take your prescriptions with higher frequency when something hurts more, you are actually A-B-(A-B)+-testing different dosage regimes.

If you are not writing down what you are doing and your observations while performing these self-experiments:

In other words, unless you do your self-experiments properly, you are probably going to waste a lot of time, effort, and money doing things inefficiently, repeatedly, and ineffectively. The only actual choice you have is between doing your self-experiments properly from the very beginning or doing them implicitly and incompetently at first and then re-doing them again at a later date. The first step of proper self-experimentation is to start writing down what you do and your observations.

Generally speaking, the number of useful facts you can infer about yourself is a polynomial of second or higher degree over amount of data you have. In the simplest case, that number is proportional to the square of the amount of collected data, since those inferences come from interactions between different self-tracking variables. In practice, however, that degree is higher because many interactions between variables generate latent variables.

Therefore, if you are unable infer anything useful from your collected data, most likely, you just don’t have enough.

Additionally, the above implies that there’s no self-tracking “sweet spot”. The time and effort you have to spend self-tracking is linear in the number of variables you track, but inference efficiency is quadratic or better. Therefore, the most efficient way to self-track is to capture as many variables as you can. At the very least, you should be capturing slightly more than you can actually analyze at the moment.

Moreover, in practice, for most self-tracking variables, the minimum amount of data you would need to infer anything useful is about two years of data. Therefore, if you plan to self-track something, you should commit to doing it for the next two years at the very least, otherwise, all that effort is probably going to be in wain.

However, as far as I can see, the vast majority of people that attempt to do this properly with even slightly onerous variables give up before that. It happens even for many of the people that try to do it seriously. In my opinion, the fact that so many people fail at this stems purely from bad methodologies and tooling, not their willpower or habit-forming issues. I’m not very good at forming new habits either, and yet I track and infer a ton of health-related stuff about myself, using zero internet-of-(spying-on-me)-things devices, apps, and web services.

I do this by applying the following principles:

Common objections

“Self-experiments!? Those are dangerous!”

If it was not clear from all of the above, the self-experiments this document advocates for are not of the kind where you drink mercury to gain immortality or expose yourself to radiation to gain superpowers. If you feel like your self-experiment might be unsafe, don’t do it! It’s that simple.

“Self-experiments!? Scientists did proper studies! You just need to read those!”

Unfortunately, the fact is, for most medical statements you can find PubMed publications that appear to support both them and their reverse. The Great Replication Crisis never stopped, most published research findings are still false.

Most medical studies ignore regression to the mean and effects of measurement, misuse already misleading p-values and confidence intervals, use absurd hypothesis choices and study setups, and fail to supply Bayesian evidence for each hypothesis they test, making meta-analyses of those studies incredibly hard. Which is why most medical meta-studies fallback to what is essentially “voting” on the result, which is not a proper statistical technique for anything except Bayesian prior probability approximation. See that note for detailed discussion of the above statements and available workarounds.

And yet, the mainstream picks sides on some of those issues with unreplicable evidence and says that the evidence is inconclusive for the rest. The unfortunate fact is that the evidence is usually inconclusive in all the cases. In other words, on many medical issues where the mainstream picks a side, it’s as likely to be wrong as not. When they say the results are inconclusive, the results very well might be very conclusive for you specifically. Unfortunately, there’s just no way to know without a self-experiment.

<<medical-malpractices> Some reminders:

On the other hand, note that while most published medical studies suck as evidence of therapeutic effectiveness, they are pretty useful as a source of hypotheses and as evidence of experimental protocol and substance safety. That is, if a study tests something applicable to you and there are lots of studies in last 10+ years that give applicable substances to hundreds of healthy human subjects in high doses, and their LD-50s are much higher than that and also higher than what you can take accidentally, and there are no known contraindications for you, then it’s probably safe to self-experiment with it. Though, obviously, you should apply common sense and start small.

Importantly, however, FDA’s (or similar agency’s) approval alone is not a good evidence of safety. To quote “Postmarket Safety Events Among Novel Therapeutics Approved by the US Food and Drug Administration Between 2001 and 2010” by Downing et al:

Among 222 novel therapeutics approved by the FDA from 2001 through 2010, 71 (32.0%) were affected by a postmarket safety event.

Or, to put it another way, on average, almost a third of FDA-approved drugs turn out to be unsafe.

“Self-experiments!? Doctors and other experts learned everything of importance already! You just need to listen to them!”

Having said all that, note that I’m not saying that mainstream science, medicine, and/or doctors are useless.

Also, therapeutically speaking, modern mainstream medicine is pretty good at solving acute issues.

However, when we move away from the acute problems and direct medical procedures, firstly, a lot of folk advice given by doctors, books, podcasts, videos, and other media does not even cite anything relevant, if they cite anything at all. They simply repeat things they heard from one another, playing a broken telephone game with each other and their audience. See the sibling note for some fun examples where supposedly evidence-based well-known folk mainstream health claims that get repeated all the time as if they are common sense actually say almost exactly the reverse of what the studies they supposedly based on actually claim.

Meanwhile, modern conveyor-belt-like doctor-patient interaction workflows common in most industrialized countries make medical error discovery, confirmation, and fixing much harder than it should be. If you ever needed to visit a bunch of doctors all at once and then tried to second-guess their advice, you are probably aware how often doctors screw up little but important details, like dosages. But, can you even call your doctor to confirm his/her instructions after your visit? Say, you do a bit of research and discover that the dosage on the prescription slip appears to be incorrect for your body mass. Are you not supposed to question it? Are you not supposed to try and confirm it? Many medical organizations make it all deliberately hard for stupid reasons.

Also, personally, when I did question it, some doctors I’ve met became pissed off at me questioning them, and then stayed pissed off even after I turned out to be right. You just have to accept that this is going to happen sometimes, relax while it is happening, and then never visit this doctor ever again.

Is it then at all surprising that, as Forbes claims:

Medical errors account for 9.5% of all deaths in the U.S. each year, making medical malpractice a leading cause of death.

and that people now regularly turn to “AI” bots for medical advice and then attempt to buy pharmacological substances with “hallucinated” names?

Secondly, much of the advice claiming to be based in science is based in bad and/or outdated science. In fact, personally, I found that literally every single piece of advice I ever received from a medical professional that was supposedly aimed towards solving a chronic problem of mine was either

Even more sadly, I actually ignored some easily available relatively well-known life interventions because the mainstream told me those claims were false and/or those interventions were dangerous. But when I tried them out, starting small and tracking any side-effects carefully, I found that I was able to fix all my leftover medical issues after some more trial and error over protocol and dosage details. No dangerous side-effects they were scaring me away with ever materialized.

Therefore, if you reached enlightenment on this issue, you are probably not listening to most of these “experts” already and you are probably already systematically checking that the things doctors tell you are not bullshit before doing or not doing them. Personally, however, I see many of my friends and family experiencing and citing this broken telephone content on the Internet all the time. And I have yet to see many of them start systematically looking up things their doctors say to them and searching for alternative opinions on those subjects, even after I point out the above issues repeatedly. There are some personal success stories there, but the level of trust people have in the medical professionals, even when they are aware of the above issues, drives me nuts.

Which is not to say that there are no good doctors, but in the modern day they are the exception, not the rule. A good doctor should be a good educator that can teach you enough specifics about how your body works, with proper scientific citations, so that the treatment plan he/she recommends would be the obvious next step. Instead, most doctors today are simply prescription-writing technicians.

When people get treated as a piece of machinery passing a medical professional on a conveyor belt, it’s not surprising they turn to more information-rich sources. The issue is, most of those information sources are also full of bullshit.

Which, again, is not to say that there are no experts giving good advice. But you can only figure out which of them are not feeding you bullshit by checking their sources, checking those sources are not bullshit too, and then properly self-experimenting with said advice.

“You want regular people to do deep scientific literature research and proper self-experiments?! This is completely unfeasible!”

Okay, but what’s the alternative?

As noted above, it’s a sad fact that most health-related studies are performed improperly and their results are false. However, even if all those studies were done properly and their results were true, said results would not necessarily apply to personally you. Different people’s are different: people with different ancestries are genetically adapted to different diets (e.g., see, PEMT gene and choline), natal and early childhood experiences can also influence person’s biochemistry for the rest of their life (e.g., see, natal carbohydrates and diabetes), etc. It would be nice if there was a way to figure out which medical treatments, diets, life interventions, productivity enhancements, and/or similar things would work for you from first principles somehow. However, in practice, self-experimenting over potentially personally-relevant hypotheses is the only way you can actually improve your life here!

Luckily, self-tracking actually becomes easier and easier and more and more effective as you setup your system to collect more and more data about yourself. Therefore, if you self-track your implicit self-experiments and other things religiously, you can actually probably automate most of this research.

For example, personally:

I find it highly unlikely that there are no useful discoveries an average layperson can make about themselves with some self-tracking and self-experimentation. Personally, I discovered something new and interesting about myself for almost every new variable I started self-tracking. Having experienced these things, I think it’s kind of inevitable that in an eventual future, an average layperson would be doing all of these things too. It’s just that for most people most of the relevant data is probably going to be collected implicitly in background by completely automated tools which do not exist yet. The hypothesis inferences, unobtrusive self-experimental suggestions, and resulting inferences are probably going to be quite close to what I’m doing myself now, however.

For example, I can totally imagine a future where most people offload their grocery and medicine shopping to virtual assistants that also time-track their humans and manage their appointments. Those assistants would then use those responsibilities to perform dietary and pharmacological experiments on their humans by tweaking the lists of things they buy, tracking what their humans consume when and in what order, and unobtrusively make lab test appointments for them. Depending on various implementation details this can be a utopia, anti-utopia, or both.

For instance, in such a future, why would you advertise to humans when humans don’t make any purchasing decisions themselves? Thus, I can totally imagine a future where there are no human-targeted advertisements anymore and all advertisements are targeting virtual assistants instead. Which can be implemented in banal, nice, and/or nefarious ways:

My discoveries

Let’s discuss some of my personal evidence for the above claims.

Low-carbohydrate diets work, high-protein high-meat diets are awesome

As noted above and shall be discussed in detail below, in 2009 I noticed that I was gaining weight almost arbitrarily sometimes and decided to do some body mass, circumference, and food diary logging, combined with periodic taking of full-body photos/selfies, to figure out why that is and to track my muscle definition visually. Then, one memorable day in 2012 I decided that I’d collected enough data and I should finally do the analysis part. So, literally the first thing I did was:

And thus, I immediately discovered that low-carbohydrate diets existed (a fact I was unaware of) and that they appeared to work for me.

I.e., it appeared that the reason I was gaining weight almost arbitrarily sometimes was because I ate more carbs at those times, even when those intervals had me following supposedly healthier diets. In fact, in one specific period I got pretty fat very quickly by eating a supposedly super-healthy vegan diet consisting of fruit, vegetables, legumes, and no animal products at all. This went against everything I was taught by nutritionists before that point. So, I tried to consciously follow a low-carb diet for a while, and it worked.

In 2026, many of the international readers of this will probably say “Duh! Obviously!”. In 2012, in Russia, when I told other people this, most of them did not even know low-carbohydrate diets existed. Neither did I, before I crunched the numbers. Most people I’d shown my data and conclusions to thought I screwed up my measurements or analysis somehow. Of the people that did hear about this low-carb stuff and were not surprised, the vast majority were horrified I actually decided to switch to such a diet full-time.

You must stop doing this! Did you not hear?! High-protein diets irreversibly damage your kidneys!

was the most common reaction.

“But mainstream nutritional advice claims that high-protein diets are dangerous!”

So, obviously, I looked that up, and, indeed, discovered a bunch of articles claiming just that. And so, I listened to that advice for a while, switching back to the supposedly “healthy” diet. But then my self-tracking data started to show deterioration again, and then I noticed that I actually started to feel objectively worse. I can’t describe how I struggled with this disparity, trying to figure out micronutrient ratios that satisfy all of these conditions. This was before I became aware of the issues with medical research, my trust in mainstream medical advice was almost absolute then.

But then, after some more time and more research, I discovered “Good Calories, Bad Calories” book by Gary Taubes, which presented evidence for low-carb diets while pointing out corruption of nutritional science by Big Food. That book did not age particularly well, but it was enough to make me start questioning these things. And thus, after I read it, I decided to switch to a low-carb diet full-time again. Though, I decided, I would also carefully log anything that could be associated with developing kidney problems. 10+ years later, no issues yet.

Note however, that in those times the mainstream considered this to be an absolutely wacky thing to do. A bunch of my relatives organized a whole campaign trying to stop me from doing these things.

How many eggs per day are you eating?! Six to twelve?! You will die from cholesterol overdose!

It was a whole thing. Even a perfectly normal lab blood work did not convince some.

Meat versus illness

Though, that was not the end of it. Even after switching to a low-carb diet I still accepted “red meat is bad for you” and “vegan diets are healthier” mainstream claims and preferred most of my protein to come from plant-based sources, whenever possible. After all, why would Big Food corrupt that research? It has nothing to do with making sugary foods appear anything but addictive poison. Also, generally speaking, meat is tastier than most vegetables, so, I thought, big fast-food chains ought to be trying to corrupt the research in the reverse direction, if anything. Also, some of those “red meat is bad” claims came from IARC who had stood up to corporate and political7 pressures before.

But then, later, I listened to Shawn Baker on Joe Rogan’s podcast where he talks about the carnivore diet (consisting of only red meat and nothing else) and its benefits, which went against those claims. “Huh”, I thought, noted it down, and forgot about it.8

But then, years later, while looking at a progression of my full-body selfies I noticed some very subtle changes that could have been taken as signs of elevated blood glucose levels. So, I bought myself a glucometer, checked blood glucose impact of everything I ate against it, discovered some things I most certainly should not eat, excluded those from my diet, and then discovered that animal products actually usually decreased my blood glucose levels compared to both plant-based foods and even my fasting baseline. Quite surprising!

Also, after the above sequence of events, I generalized and systematized my occasional taking of full-body selfies into its own proper self-tracking method I decided to call “medical selfies”.

Then, later, I got a prolonged respiratory tract bacterial infection which nevertheless was mild enough I could experiment with it, so I tried modifying my diet in various ways while keeping track of any visible inflammation in the mouth via medical selfies and the sum of sizes of my palpable lymph nodes, trying to see if there were any easily noticeable effects. (This was before I started mega-dosing with Vitamin C.) And then I remembered about Shawn Baker’s claims, and, surprisingly, consuming 500 grams (1.25 lbs) or more of red meat per day increased my speed of improvement significantly. That result survived a single short explicit A-B-A-B self-experiment, too. And replacing red meat with an equivalent amount of chicken meat produced no discernible result at all.

A single short experiment did not convince me that this was a true association, but, taken together, the above observations went against literally every single bit of mainstream nutritional advice I was aware of (since a high-red-meat diet implies a high-saturated-fats diet, which was supposed to be awful too), which made me quite suspicious. Luckily, by that time my medical self-tracking became quite advanced, so I could actually start testing those suspicions for myself.

Thus, I defined my illness vector as a vector of “body temperature in degrees Celsius minus 37.0 (or 0 if that value is negative) plus 1 if feeling ill (or 0 otherwise); the sum total of the number of times I coughed and sneezed recently” and then went to check if in my medical logs “carnivority” versus “veganness” of my diet was correlated with values of my illness vectors. This, I have to note, was rather annoying to do, since I had to manually tag all the foods in my log, because nutritional labels are not at all helpful for this. But as a result I found that, yes, the closer my diet was to “vegan”, the more time I was spending ill or ill-ish. Admittedly, the regression coefficient was not huge, but still.

My first hypothesis explaining this effect was that it came from the antibiotics that livestock gets stuffed with. Eat more meat, get more antibiotics as a result, get less illnesses. Makes sense, right? But a simple calculation with the data from a relevant study showed that even a diet of 100% red meat provides a dose that is 1000–10000 times too small.9

My second hypothesis was that it comes from Vitamin D, which immune cells are known to be receptive to (in a way science does not at all understand at the moment). Animal sources are rich with it, and plants are not. But correlation between my Vitamin D supplementation and my illnesses was almost non-existent. (Admittedly, a proper calculation here should have taken dietary Vitamin D and sunny Vitamin D into account too, but I found no good data source for the former and the computing of the latter needed more infrastructure I did not have yet.)

But, well, I thought, maybe that batch of red meat I self-experimented with had 1000 times more of the antibiotics than usual, or maybe it was especially rich in Vitamin D, who knows. So, more self-experimentation with meaty diets was in order.

“But mainstream nutritional advice claims that eating lots of red meat is bad for you for multiple reasons!”

But, according tho mainstream advice, that would’ve made my further meaty self-experiments quite risky. Given the contrast between the mainstream claims and the above observations, however, after a bunch of research on the subject of dissenting views, I decided to just go and do proper meta-analysis of the evidence for those claims myself, as I should have done from the very beginning.

In the end, I actually ended up spending months doing this, the results of that are discussed in a sibling note. Note that many of the conclusions in that note are rather subtle and even a simple list of statements with no explanations at all takes several pages. But, to give a very short summary on the abovementioned issues:

In other words, I now think that mainstream nutritional advice is actively harmful!

Also, methodologically speaking, the main result of all that meta-research is that I now disregard mainstream nutritional advice and mainstream dietary reference intake tables. Instead, if I have a nutritional question I can’t answer from my own data, I now check studies that use astronauts as subjects. Space agencies have essentially complete control over astronauts’ diets and schedules, so the quality of data they use for their research is much higher.

Results

So, having convinced myself that all of the abovementioned mainstream claims are bullshit I decided to test various high-meat diets to see if they would improve things. As a result of my self-experimentation I discovered that even the 100% red meat carnivore diet advertised by Shawn Baker is quite feasible:

Additionally, I discovered than on high-meat very-low-carb diets (which includes the carnivore diet):

Additionally, I discovered that high-meat non-very-low-carb diets (which excludes all of the above) are awful because:

See the sibling note for more discussion on this subject.

Butter can help with acid reflux

Many years ago I started getting acid reflux occasionally. I went to a doctor, had all the tests done, got prescribed a bunch of medicines and a diet. If you look at that the page behind the above link, you can get the general idea of dietary recommendations there. In particular, note that fatty foods are banned.

So, I was following that diet for a while, but according to my tracking it did not make a difference, so I abandoned it eventually and switched to medication-only management.

But then, after I switched to low-carb diets, analyzing my food diary logs one day I noticed that I got less acid reflux when I was eating more butter. According to the mainstream theory, this is not how it is supposed to work, it should cause more acid reflux, not less. But I tried to do it on purpose, and it did help.

For years after that, I was managing mild episodes of acid reflux by simply chewing a huge piece of butter every evening before bed. I.e., I did exactly the reverse of what my doctor told me to do, and it improved things. (I did still need medication for the intense episodes.)

Then I also noticed that some of the chairs I tried while self-experimenting with lower-back pain improved my acid-reflux situation too. Moreover, mega-dosing with Vitamin C essentially resolved it.

Mega-dosing with Vitamin C works, in most unexpected ways

My relationship with high-dose Vitamin C supplementation is very similar to the low-carb diets story discussed above.

I noticed that Vitamin C intake was present in my regression coefficients table for my illness vector immediately on its generation. But at that time the mainstream medicine taught me that taking more than 250 milligrams per day was useless and more than 3 grams (3000 milligrams) per day was dangerous and would cause nausea, diarrhea, and kidney stones.10 So, I very rarely supplemented with Vitamin C, 250 mg/day is pretty easy to get from food (from bell peppers or kiwi fruit, for example). That was until, after getting vaccinated multiple times, I got infected with COVID-19, experienced a bunch of radical life changes, had an awful flu episode, and started getting bacterial upper respiratory tract infections more frequently afterwards. (Which of the changes, if any of them, is the origin of the issue I have no idea yet.)

Therefore, I did some research on the subject and collected a large list of potentially beneficial therapies. One of those was mega-dosing with Vitamin C, which I heard of before, but dismissed at the time. So, this time I decided to research it properly. While doing that I discovered the following pertinent Vitamin C facts.

Vitamin C actually has many different forms, the two most common of which are:

Additionally, these substances can be taken/delivered in several distinct ways:

Finally, I found some relevant claims on its effectiveness:

As noted above, I actually heard about some of those claims many years earlier and dismissed all of them at the time. Firstly, before doing all that research I was unaware of sodium ascorbate and so the precise mechanisms by which ascorbic acid could be safely taken in high doses/concentrations confused me. Secondly, Thomas E. Levy’s claims, which I only heard as a retelling, seemed too good to be true. Thirdly, Suzanne Humphries was also easy to dismiss, she was a fan of homeopathy at the time, the point of view she abandoned since.12 Finally, “House, M.D.”, which I loved at the time, had an episode mocking high-dose Vitamin C treatments.

Meanwhile, it appears that the most relevant claim of Thomas E. Levy, which is that high-dose intravenous sodium ascorbate is effective against infections and sepsis, was essentially accepted by mainstream since. See, e.g., there, there, or most other recent articles mentioning “sodium ascorbate” on PubMed. Though, according to Levy’s writings, if the linked studies gave even higher doses, the effects would have been even better.

Also, an older study claims that supplementing with Vitamin C regularly cuts length of common cold episodes 8% for adults and 14% for children, but it makes no effect on prevention, and no effect if taken after the symptoms appear. In the abovementioned presentation Suzanne Humphries’ essentially argues that this is because:

And thus, she argues, taking mega-doses of sodium ascorbate should be helpful, the higher the dose, the better the effect.

She also claims that low-concentration ascorbic acid solution sprayed onto skin helps with many skin conditions.

And thus, I decided that sodium ascorbate would be a good first candidate for self-experimentation on my “more frequently ill” issue, given its high LD-50 and that I didn’t have any counter-indications to it. Therefore, I decided to test

If you plan to self-experiment with this too, be sure to check out the abovementioned presentation for possible side-effects, medical conditions that are incompatible with mega-dosing, and dosage calculations.

So, I started small, with 1000 milligrams of sodium ascorbate per day, then gradually grew the dose, first to the sometimes recommended 100 milligrams per kg body mass per day, then as high as I could get it, noting down any side-effects, then backed off to the dose that made me feel best.

Firstly, as noted above, I noticed almost immediately that a glucometer can be used for blood concentration measurements. Sure, it does not give absolute values, but it is pretty useful for figuring out dose and delivery method to blood concentration relationships. To do this properly I switched to the carnivore diet for that self-experimental period, which made my blood glucose levels essentially constant, thus making any changes to the measurements there to be Vitamin C-related ones.

Secondly, I discovered that Humphries’ recommended way to measure the optimal dose of “take more sodium ascorbate until your bowels become really rumbly and/or you get diarrhea, then take 90% of that the next day” didn’t work with the carnivore diet. I hypothesize that it simply didn’t leave that much available water in the bowels for that method to work reliably. Which is why I started tracking saturation by it spilling into urine instead: it changes color at first, then becomes cloudy at very high concentrations.

Thirdly, using the above measurement method, I discovered that, compared to other delivery methods, sodium ascorbate and/or ascorbic acid taken as dry powder and/or capsules are ineffective. Sodium ascorbate and ascorbic acid powders and capsules are cheap, but, evidently, they mostly end up in stool instead of blood. Capsules also prevent some of sodium ascorbate’s other beneficial effects, which shall be discussed below. Also, ascorbic acid, including in its liposomal form, can’t be taken in high doses even in capsules. However, I found that dissolving dry sodium ascorbate powder in water and then taking regular small sips of the resulting solution produces very good results.

Fourthly, I found that I could take more of it and raise my blood concentration much higher if I did not drink any more liquid than usual. In other words, taking sodium ascorbate by dissolving it in seemingly reasonable amounts of water first simply made me pee it out almost immediately. Dissolving it in small amounts of water and replacing some of my normal daily water intake with that worked the best. This is the method which allowed me to reach the probably unhealthily high concentrations that made my urine cloudy.

Finally, I discovered that the best way to both raise blood concentration and prevent abdominal cramps and diarrhea (on my normal low-carb diet) and nausea (on both low-carb and carnivore diets) is to stagger oral doses as much as possible. For me, abdominal cramps begin with about 3 grams all at once or close-enough, nausea and then diarrhea appear with more than 6 grams all at once or close-enough, if not on the first attempt, then on the second. However, the maximum I managed to take in a single day is actually slightly over 70 grams. I did this by taking 500 milligrams approximately every 7 minutes for about 16 hours. This made me feel bloaty, but not sick or nauseous. Though, note that bowels have to adapt to higher doses of sodium ascorbate, so to do the above I had to grow the daily dose gradually.

In other words, I discovered that Wikipedia’s summation of relevant studies of:

Oral intake of dietary supplements vitamin C in excess of requirements is poorly absorbed, and excess amounts in the blood are rapidly excreted in the urine, so it exhibits low acute toxicity. More than two to three grams, consumed orally, may cause nausea, abdominal cramps and diarrhea.

is incorrect. I found that it’s very easy to take sodium ascorbate orally in doses of 30 grams per day or less and little of it ends up in urine if you stagger your doses well enough. Also, it can be taken in even higher doses, but doing that needs more precision and care than most people could probably regularly manage.

How to take it most effectively

As noted above, the most effective way to take sodium ascorbate powder is to completely dissolve it in water, which greatly improves its absorption, and then sip that water in the most staggered way possible, which both improves absorption and eliminates many side-effects. However, note that sodium ascorbate dissolved in water starts oxidizing (slowly), gradually changing color to orange and loosing potency. So, dissolving too large of a dose all at once is actually counter-productive.

A teaspoon of water is about 5 milliliters, a standard cup of water is about 250 milliliters, which is about 50 teaspoons. So, if you drop 25 grams of dry sodium ascorbate into a cup, fill it with water, stir it until all of the powder dissolves (which could take a minute of stirring or more), as a result, you will get about 50 teaspoons of 500 milligrams of dissolved sodium ascorbate each. To take that most efficiently, you would need to take a teaspoon every 19 minutes for 16 hours. Unless your room temperature is unreasonably high, almost none of it will oxidize in the cup in those 16 hours too.

Similarly, if you drop 70 grams of dry sodium ascorbate into 700 milliliters of water, you would get about 140 teaspoons with the same dose. A teaspoon every 7 minutes for 16 hours is 137 teaspoons. This is how I managed to take my largest daily dose when self-experimenting.

So, in practice, you can simply make such a cup of sodium ascorbate solution in the morning, put in on your work table, and sip it throughout the day, using an automatically resetting timer as a reminder to do it. This works pretty well, except:

Meanwhile, twice an hour for 16 hours is 32 sips. So, what I actually do at the moment of writing is I take a cup that is slightly over 320 milliliters, drop whatever daily dose I want into there in the morning, fill it up with water, stir, take the spoon out, put that cup on my work table, away from direct sunlight, and take ~10 milliliter sips every ~30 minutes. A 10 milliliter sip is about my normal “tiny sip”. After a bit of self-training I don’t even need an alarm anymore.

Unexpected effects

The results of the above self-experiment shall be discussed in the following subsection. However, what I would like to note first is that after I started mega-dosing with sodium ascorbate, some issues I thought to be permanent chronic conditions of mine suddenly improved greatly or just vanished completely. Notable examples include:

I verified the above effects with A-B-(A-B)+ self-experiments too: stopping sodium ascorbate for two weeks made all of the above issues return (in milder forms, but still), taking it again made those issues vanish again.

Note that bleeding gums and mild depression/persistent fatigue are symptoms of scurvy. I ate citrus fruits and sauerkraut regularly for most of my life and yet mega-dosing for a week produced incredible improvements. Which, to me, implies that “scurvy” is actually a set of separate conditions, each with its own Vitamin C requirements.

The anti-depressant effect alone is worth taking sodium ascorbate for the rest of my life. Which I now plan to do.

Additionally:

These effect did not revert in any obvious way after two weeks of no sodium ascorbate. But I did not try waiting longer for obvious reasons.

Advertised effects

So, with the above discoveries I essentially abandoned some of the original self-experiment, since I’m no longer willing to not supplement with less than 100 mg/kg/day of sodium ascorbate. Meanwhile, while supplementing with that or more I observed the following:

Takeaways

I could have started regularly supplementing with sodium ascorbate 10 years earlier if I’d trusted my regression tables more than what people said. Which is sad, quite literally, as it improved my average moods significantly.

Nutritional synergies

Note that many of the things discussed above are synergistic:

I think these effects are a good hint that all of the above should be done together.

Changing clothes more frequently reduces frequency of being ill

For a fun unexpected and very idiosyncratic example of self-tracking inference, one day I was browsing a table of regression coefficients for my illness vector computed against all other things I was tracking and I noticed that one of the top things that reduced that metric was the average number of times I was changing clothes per day. (Which was a thing I started tracking separately by accident, I was trying to improve predictive power of my task switcher.)

At first I thought it was leakage from exercise or simple bathing activities, since I usually change clothes after. But then it raised a question why I have not noticed that correlation before I separated clothes changing task away from the rest. Also, that correlation persisted even when all of those linked events were removed from consideration. So, I thought, it must be a junk correlation-not-causation thing, and I ignored it for quite a while.

But that correlation only grew with more data, reminding me about it each time I looked at those regression coefficients. And then, one fateful hot and sweaty day — as I was logging a sneeze event with a key-combo and subconsciously thinking something in the line of “Ugh, where did that draft come from? Hmm, also, I notice I’m sweaty. I should go change my shirt!” — the obvious explanation struck me.

Apparently, I have a bit of hyperhidrosis localized to my back and feet. (I had it since childhood and I thought this was normal before I looked up “being excessively sweaty” while figuring out this correlation.) This means that when I’m slightly hot and thus slightly sweatier than I should ideally be, I start experiencing micro-hypothermia episodes when sudden drafts of air grab moisture from my sweaty clothes. Which can be quite common in a room with AC on or when moving sharply. Changing clothes frequently fixes that.

And thus, I now simply change clothes multiple times a day, as soon as I notice I’m sweaty, and on a timed reminder. “It’s fine, I’ll dry out eventually anyway!” no longer. In summer, taking several sets of shirts and pants, wearing one of them while drying the others, and then round-robin switching them multiple times a day helps quite a bit. In winter, doing a similar thing with socks helps the most because in that season vertical room temperature gradient is much larger, which could cause my feet to start experiencing micro-flash-freeze episodes on every step in slightly sweaty socks.

I probably would have never consciously learned about this and would have never thought of doing such things to fix it if not for self-tracking. (Clearly, I did occasionally perform that fix unconsciously, otherwise the correlation would not have been so apparent.) How would you even think to ask the web for such a thing? How many other people even have such an issue? (Wikipedia claims any type of hyperhidrosis affects less than 3% of people, but my specific case appears to be especially uncommon.)

Also, note that these life interventions were derived from a correlation between variables which had to be tracked manually. No commercial gadget could have helped me here.

Changing clothes more frequently helps with lower back pain

But then, after doing the above shirt+pants-changing thing religiously for a couple of years, I noticed that it also reduced episodes of lower back pain.

Which I then debugged away completely by experimenting with different chairs.

Pure water is more effective against conjunctivitis than anti-microbial eye drops, if used early enough

I became very susceptible to conjunctivitis (pink-eye) after I got a tiny piece of construction debris into my eye in 2010. Just wiping my face with a clean towel that was laying about in a closet and was not washed recently enough would be enough sometimes. It was super-annoying. I was prescribed eye drops for it and I had to use them enough that I actually became allergic to one kind of them eventually and had to change prescriptions at one point.

But via self-tracking I discovered that I had no episodes of conjunctivitis in periods of time where I was showering and washing my face very frequently (because of the weather). Therefore, it made total sense to me to check if simply washing an inflamed eye with clean water the next time I get conjunctivitis would help too. And it actually worked, stopping the inflammation outright.

With some more self-experimentation I discovered that when I washed the eye just as the symptoms started, clean water worked better than eye drops. Which, when I think about it, makes total sense. A very generous eye dropping session puts maybe 2 milliliters of liquid into an eye. Therefore, 250 milliliter cup of water is at least 125 times more effective at washing out the offending bacteria. (Though, when I wasn’t fast enough and the bacteria got to spread, I needed the drops anyway.)

It’s absurd none of the doctors I visited ever even mentioned any of that. all of them went directly to prescribing me those eye-drops. But the above cut my eye drop use 20x immediately.

In all the places where I lived since, tap water was clean enough for it, but your mileage may vary. And being a bit paranoid about it, I usually preferred to have some boiled water in a jar for this. In winter I would usually add a bit of freshly boiled water into there, to prevent eye hypothermia, just in case.

There’s actually a well-known traditional remedy to conjunctivitis, which is to wash it out with freshly brewed warm tea. It works too. But, it always took me quite a bit of work to make a fresh pot of tea and then cool it enough so that I could use it for eye washing. Meanwhile I was doing that, bacteria were multiplying eagerly. Which is why, I suspect, simple water was actually way more effective.

Pure water helps with curing chronic conjunctivitis

Since then I debugged that problem away completely by amply washing my face twice a day, changing my pillow’s case once a week or more, and ensuring that my hair is always clean enough and is never over my eyes (i.e., it’s cut short or tied away). But these interventions, too, make total sense, since those are the most common ways dirt can get into an eye indirectly.

I did not need any eye drops since I started doing all of that.

Some types of chairs reduce frequency of lower back pain

So, after I fixed most of my chronic lower back pain by reducing sweatiness by changing shirts frequently I decided to debug away the rest of it by experimenting with various chairs, because I noticed that using some chairs appeared to improve things compared to others.

So, after years of self-experimentation I found that:

In any case, five years and counting of no back pain at all since I started doing the shirt+pants-changing thing and using a proper chair together.

Some types of chairs help with acid reflux

Interestingly, when I started using chairs satisfying the abovementioned conditions, it also reduced the frequency of my acid reflux episodes.

That is, sitting in a proper pose also reduced my acid reflux.

Which, when I think about it, also makes sense, as straight sitting position places the lower esophageal sphincter as far away from the stomach contents as physically possible.

Workspace configuration tracking

The above observations made me also start tracking and self-experimenting with workspace configurations.

For instance, apparently, the best chair placement for me is with its back against a wall. This way I can’t move my chair backwards, consciously or not, the desk-edge-to-chair distance becomes fixed, which also influences my posture in a positive way. (Though, it makes getting in and out of the chair slightly annoying.)

Also, thinking about this, I have to note that the classical western office/study setup also adheres to this principle by making the classical “cabinet” chair have a breathing straight back, and making it heavy to prevent easy movement. See a drawing from 1719 and a photo from 1903. Meanwhile, most modern office chairs are light and have wheels, which makes it very easy to move such a chair backward and slouch in it, which is evil. So, our forefathers did it properly, but modern material science made worse solutions more convenient, apparently.

So far, other effects I discovered while self-tracking my workspace are very marginal.

Bidet/bidet shower

From among the health-related things many people are aware I consider bidet/bidet shower to be the most underappreciated thing.

You might be from a culture that considers this common sense or even from a country that requires all bathrooms to have one by law. But if not, then you should add to your bathroom the next time you renovate, start using it, make it a habit, and you’ll never go back. The effects of regular use of this thing were a bit of a revelation to me. You can look them up in the above Wikipedia articles. I confirmed most of those effects on myself and would rate the results to be 10/10.

The use of these devices will also reduce your toilet paper use to almost zero, which is both more environmentally friendly and cheaper.

Anti-procrastination

Even without looking at my self-tracking data I’d noticed that my moods were consistently worse in winter.

So, one day I was reading about seasonal affective disorder and light therapy after reading a section of “Inadequacy and Modesty” by Eliezer Yudkowsky (2017) on the subject, and I thought I should probably check if adding astronomical calculations to my regression computations produces anything interesting. By playing with that analysis I found that not just my moods, but my overall productivity, as measured by the true task time spent doing things I conscientiously want to be doing (not what my “Instant Gratification Monkey” wants me to be doing instead), i.e. the reverse of procrastination, is very daylight-dependent. This discovery was quite surprising to me. Before performing those computations I was quite sure that

But, apparently not.

To take advantage of this observation I had to force myself to become a “lark”. So far, this is the hardest behavioral modification I managed to implement. To force myself into this I had to make a bunch of inference, scheduling, and notification tools, and then learn to use them properly to radically alter my (pre-)sleeping habits. I have to spend constant effort to maintain these changes to my daily schedules still, but my self-tracking self-experimental data shows those changes are effective, so I’m pretty happy with the result.

Additionally, I’ve been re-running the above analysis regularly ever since, to track any changes there. Those regression tables are quite informative. At the moment of writing, my regression coefficients show me that my average daily productivity is most dependent on, in order:

Some of these dependencies were quite surprising when I first discovered them.

Self-optimization

In short, most of the above can be summarized as follows. I found that I can rarely replicate the results of most health, nutrition, and productivity studies over myself. Instead, I found that the most reliable way to actually improve my life is to

I also found this way of doing things to be quite efficient and effective, especially after I automated most of it away. Also, this approach has a benefit of needing very little scientific literature research to use effectively. Which is to say, it’s a simply superior method of self-optimization, at least when compared to trying to use advice given by people that did not study your specific situation first.

Now, before discussing the particulars of how I setup my personal exocortex software system with all of its features and use those features to self-track everything, analyze my data, and self-experiment, let’s discuss the general approach to these things. In other words, let’s summarize the contents of “Common pitfalls” and “Common objections” sections in a positive way, as a series of explicit steps you can follow to apply the scientific method to your life.

Thus, to self-optimize your own self, you should apply the following general algorithm:

Privacy and future-proofing

Note how the above calls for private and future-proof data collection. Let’s discuss how you can setup your personal exocortex software system to have these properties.

Prerequisites

The UNIX way and UNIX-like systems

The UNIX way is a philosophy which says that a good software system should be implemented as a well-integrated collection of simple programs/commands each of which does its own thing (well), as opposed to a huge system that does a lot of things all at once (poorly). More formally, the UNIX way is a design philosophy that says that user-facing systems should be implemented to

UNIX-like systems, a.k.a. POSIX systems, are systems that follow the UNIX way in their design and implementation.

To take full advantage of a UNIX-like system, typically, its user needs to learn to use its CLI shell interface, not its GUI. This is because most modern GUIs expose actions, usually by binding them to buttons, menus, and such, but they don’t expose their functions. There are alternative GUI system design approaches that expose their functions and allow users to compose and build pipelines out of them, which allows a GUI-based system to be UNIX-like, but no popular general-purpose GUI system does this. On the other hand, this is pretty common in sound-processing software, see, e.g., the UI of LMMS. However, it is entirely possible to design a generic GUI system than does this throughout, see, e.g., “Tangible Functional Programming” by Conal Elliott (2007).

Typically, CLI-shell-based systems are considered to be harder to learn and use than the GUI-based ones, but shell-based systems are more expressive, more modular, and easier to use on ad-hoc basis, making them easy to integrate with other tools. This means that CLI tools are probably going to stay relevant as long as computing exists, which is not so obvious with GUI tools.

For example, CLI interfaces were brought back into the news cycle recently because tool-using large language models (LLMs) can now bridge the gap between the natural language and the shell. Most of what personal “AI” assistants do now is translate user’s chat requests into verbose shell scripts, run them, and then summarize their outputs.

So, I find it entirely possible that, in the future, most laypersons will use their computers via chat/voice and modern action-based modal GUIs will be relegated to history.

Operating systems

Android forks

To solve the abovementioned Android issues, independent developers started making alternative versions of Android almost immediately after its first release in 2008. These alternative Android versions are known as Android forks. At the moment of writing of this document there are dozens of Android fork in existence, of those the following ones are most notable:

Package managers

This section is a work in progress.

GNU/Linux package managers and distributions

This section is a work in progress.

This section is a work in progress.

F-Droid and its marketplace apps

This section is a work in progress.

GNU/Linux package management

This section is a work in progress.

Packages

This section is a work in progress.

My operating system usage history

This section is a work in progress.

Setup Android

This section is a work in progress.

Re-flash your device, if possible

This section is a work in progress.

Learn to use F-Droid

This section is a work in progress.

Replace or isolate evil apps

This section is a work in progress.

Learn to use Obtanium

This section is a work in progress.

Reduce tracking

This section is a work in progress.

Setup Windows

This section is a work in progress.

Setup POSIX OSes

This section is a work in progress.

Setup profile.nix

This section is a work in progress.

Setup clocks

This section is a work in progress.

Setup NTP

This section is a work in progress.

Set system clocks to UTC or a similar consistent time zone without daylight saving time

This section is a work in progress.

Otherwise

This section is a work in progress.

Setup file synchronization

This section is a work in progress.

Setup backups!

This section is a work in progress.

Productivity

This section is a work in progress.

Fix your biochemistry

This section is a work in progress.

Improve your efficacy

This section is a work in progress.

This section is a work in progress.

Setup shell

This section is a work in progress.

Learn to use version control

This section is a work in progress.

Learn to touch-type

This section is a work in progress.

Setup dmenu, emoji input, global bookmarks, etc

This section is a work in progress.

Learn to use your task-specific tools

This section is a work in progress.

Reuse time

This section is a work in progress.

TTS is magic

This section is a work in progress.

Reading is the new Latin, but not all hope is lost with TTS

This section is a work in progress.

Setup TTS

This section is a work in progress.

Untether yourself

This section is a work in progress.

Setup document archival

This section is a work in progress.

Setup document playback

This section is a work in progress.

Pipeline tasks

This section is a work in progress.

Batch tasks

This section is a work in progress.

Reuse task shadows

This section is a work in progress.

Minimize interruptions

This section is a work in progress.

Getting Things Eliminated technique

This section is a work in progress.

Scheduling

This section is a work in progress.

Minimal effective self-tracking

This section is a work in progress.

Prerequisites

This section is a work in progress.

Capture medical selfies

This section is a work in progress.

Setup capture of medical selfies

This section is a work in progress.

Setup capture of medical selfies on Android

This section is a work in progress.

Capture bust-waist-hip circumference measurements

This section is a work in progress.

Setup capture of bust-waist-hip circumferences

This section is a work in progress.

Setup capture of bust-waist-hip circumferences on Android

This section is a work in progress.

Start capturing annotations and ratings/scores/comparisons

This section is a work in progress.

Self-tracking

This section is a work in progress.

Prerequisites

This section is a work in progress.

GNU Emacs

This section is a work in progress.

Emacs org-mode

This section is a work in progress.

*.org file syntax

This section is a work in progress.

Important features of Emacs org-mode

This section is a work in progress.

Quirks of Emacs org-mode

This section is a work in progress.

Why use org files?

This section is a work in progress.

Extensions to Emacs org-mode

This section is a work in progress.

Tools adjacent to Emacs org-mode

This section is a work in progress.

Emacs calc and its algebraic syntax

This section is a work in progress.

Alternatives to Emacs and Emacs org-mode

This section is a work in progress.

My personal exocortex software system history

This section is a work in progress.

*.org logistics

This section is a work in progress.

Setup Emacs org-mode and org-alert

This section is a work in progress.

Variables

This section is a work in progress.

This section is a work in progress.

Setup Orgzly Revived

This section is a work in progress.

Setup org files synchronization

This section is a work in progress.

Setup org lensing

This section is a work in progress.

Appointments and habits

This section is a work in progress.

Exercise diary

This section is a work in progress.

Wastebin diary

This section is a work in progress.

Medical diary

This section is a work in progress.

Quantified medical selfies

This section is a work in progress.

Waist-hip measurements

This section is a work in progress.

Expenses

This section is a work in progress.

Time-tracking

This section is a work in progress.

How to structure your org files properly

This section is a work in progress.

Optimize for consistency

This section is a work in progress.

Keep a changelog of major changes

This section is a work in progress.

Name stuff properly

This section is a work in progress.

Optimize for recognizability in org-agenda

This section is a work in progress.

Organize your org header hierarchies properly

This section is a work in progress.

Don’t use tables for self-tracking

This section is a work in progress.

Be succinct, but not too succinct

This section is a work in progress.

Optimize for org-capture speed

This section is a work in progress.

Reuse self-tracking event headers

This section is a work in progress.

A personal wiki on steroids

This section is a work in progress.

How to take good notes

This section is a work in progress.

Data analysis

This section is a work in progress.

Prerequisites

This section is a work in progress.

Linear regressions

This section is a work in progress.

Latent variables

This section is a work in progress.

Correlations

This section is a work in progress.

Associations

This section is a work in progress.

Data fixups

This section is a work in progress.

Inferring things

This section is a work in progress.

Software mentioned in passing

This section is a work in progress.


  1. That is to say, modern smartphones, smartwatches, and other wearable computers suffer from the following issues that make them currently unsuitable for exocortex-like usages:

    • Most such devices have very high device-to-user but very low user-to-device IO bandwidth.

      In other words, most modern manufacturers seem to be making devices for media consumption, not productive work.

      Some kind of high-bandwidth input device needs to become ubiquitous and reach economies of scale state first.

      A neural brain-machine interface would be cool, sure, but a chorded keyboard (which can be implemented on some of the modern gaming handhelds by chording with joystick buttons) or subvocal recognition (but probably not normal speech-to-text), could also work.

    • Most such devices simply don’t have enough on-board storage.

      Most modern manufacturers actually do this on purpose, producing devices with very little on-board storage to up-sell their users on “Cloud” storage subscriptions.

      Meanwhile, in my experience, a personal exocortex software system only really becomes useful after you dump 2 TiB or more of supplementary data and indexes into it (possibly multiplied by the number of languages you speak/use).

    • As shall be discussed below, a good personal exocortex software system must be integrated into the OS, such a system is much less useful as “just another app”. Meanwhile, from a technical standpoint, the number of different boot sequences and devicetrees modern mobile devices use makes multi-device OS distribution development unfeasible.

      Something like IBM PC’s BIOS and PCI need to be standardized first.

      Which is why I think that if such a device ever appears, it would be based on x86 (most likely) or RISC-V (less likely), but probably not ARM, because literally every single manufacturer does their own thing with the latter.

      For example, did you know that on Raspberry Pi it’s actually the GPU that boots first, it then enables the SDRAM, loads the Linux kernel from the SD card into SDRAM, and starts the ARM CPU?

    So, given all of the above, if I were to speculate, I would guess that a first proper exocortex-like device would be a handheld gaming PC (like a descendant of the Steam Deck or something similar running SteamOS) married to an optical HMD glasses, initially used for AR/VR gaming. But then, somebody would develop a practical wearable high-bandwidth input device, like gloves that could be used as a keyboard, a face mask with embedded electrodes that could be used for subvocal recognition, or some such. And then, a custom “exocortexOS” GNU/Linux distro for such a device stack would be born.↩︎

  2. Любищев, Александр Александрович. “Такая добровольная каторга”. Химия и Жизнь. №12. 1976.↩︎

  3. Who is an ex-student of mine, actually. It appears that self-tracking memes are surprisingly contagious.↩︎

  4. The “law” is in quotes because while observations that work most of the time are frequently called “laws” in economics, I refuse to follow this terminology unironically. If economics wants to become a proper science, it should stop slandering mathematical and physical laws by switching to proper scientific terminology first.↩︎

  5. Note that it can be a poisoned open-weight model, not just a closed model accessed via a remote API.↩︎

  6. Actually, are such things even “advertisements” anymore?

    Say, you take an LLM, give it a list of things you like, and then ask it to suggest you something similar. However, the maker/runner of said LLM tweaks it to answer such questions with product names other companies bought “ad spots” for. So, it answers with one of those product names instead of doing a honest Word2vec/semantic search and answer generation.

    “Advertisement” is when you see a mention a possibly relevant product while experiencing something else. In the above, the model simply lies when answering a question. Is the above behaviour still “advertisement”, when no advertisement actually happens?↩︎

  7. The title of this article is misleading. It makes no claims of corruption or lobbying. The issue there is that an IARC-associated scientist went to give a talk at a subject-relevant conference, but, apparently, at a “wrong” one.↩︎

  8. Joe Rogan’s podcast has a rather large proportion of guests who are very eloquent yet talk complete bullshit. Which is why I like it. Both as a mental training aid and as a way to pierce my information bubbles.↩︎

  9. Though, out of that context, those concentrations are absurdly high. In most first-world countries, to prevent development of antibiotic-resistant bacteria, humans require a prescription to take less than 10 grams of antibiotics, typically distributed over a week. Meanwhile, cows — which we share most bacteria with — get stuffed with daily doses of antibiotics measured in shovels, for most of their lives.↩︎

  10. Why do they always try to scare you away with kidney problems?!↩︎

  11. Since then Suzanne Humphries also visited Joe Rogan’s podcast, but that talk only mentions Vitamin C tangentially, it’s mostly about vaccines.↩︎

  12. Personally, I think people should be lauded for changing their minds towards things that are much more likely to be true. Homeopathy is very, very unlikely to be true. It’s shameful that supposedly Rational Wiki misrepresents her later claims and low-key mocks her for changing her mind. There could be nothing more rational than changing your mind when you think you are wrong. Shame on them.

    Some people also point out that Suzanne Humphries is now an “anti-vaxxer”, as per “Dissolving Illusions” book by Suzanne Humphries and Roman Bystrianyk (2013), and so her current ideas should be discounted too. However, that book essentially claims that “most vaccines administered in the U.S. have negative effectiveness” and “most vaccines administered outside the U.S. are, at the very least, ineffective”. Unlike “homeopathy is effective” statement, these statements do not invoke supernatural effects, so they have much higher prior probabilities.

    On the first statement, did you know that, in the U.S., Big Pharma has a complete immunity against all liabilities for all vaccine injuries since 1986? Because of that bill the majority of modern day’s vaccines administered in the U.S. are barely effective as vaccines and instead increase their measured immunologic effectiveness by adding compounds toxic to humans (a.k.a. “adjuvants”). The use of adjuvants is outright banned in many other countries. So, the first claim I would rate as “likely true”.

    Meanwhile, on the second statement, “Dissolving Illusions” book points out how much politics, data manipulation, and outright data censorship there is in vaccine-related science and statistics. So, whether most non-toxic vaccines are actually effective or all of those effects actually come from modern day’s high levels of hygiene, high quality plumbing, and high nutritional intake is a really good question. The fact that most vaccines only ever get checked for safety and immediate immune response, not their long-term illness-prevention effectiveness is kind of absurd.

    So, personally, I think it quite likely that U.S. “anti-vaxxers” are actually correct in that most U.S. vaccines do more harm than good. Whether that’s true outside the U.S. in countries where people can sue for vaccine injuries, especially in countries where toxic adjuvants are banned, I don’t know. Personally, I’d say it would be nice if somebody actually checked, especially since it’s something that’s actually pretty easy to check.↩︎


Do you want me to complete this sooner?

Support me via: