Two very difficult health problems

The Snapshot System, like all observer-only systems, aim to help with two hard problems:

1. When an individual has a disease or health anomaly, it should be caught early and treated with maximum information, however, there are many serious problems related to detecting problems early. We will call this the Early Detection Problem.

2. Medical science is ignorant in many fields because good empirical evidence about populations either has not been collected, or, there are ethical and practical barriers to collecting empirical evidence. We will call this the Lack of Evidence Problem.

These two problems are explored in detail later in this article.

The Proposal

A Medical Snapshot System is run by an organisation that enters long-term contracts with individuals to observe them regularly using tests across everything that can be readily measured — blood, hormones, imaging, genetics, mental health and more. In principle, the organisation has only two duties of care:

There is no other duty. Most especially, there is no duty to notify anyone of any observation. This is the point where most people say wait a minute, what if they discover cancer?" That is the point everyone involved in observer-only systems wrestles with, and there are many solutions.

I often explain this in terms of a carwash. A driver pays to get their car washed, knowing the valet staff will see any rust, dents and worn tyres. But the driver is buying a valeting service, not a garage service, and doesn’t expect to receive a checklist of all the mechnical problems noticed. The driver is getting what they want, and discussing rust isn’t part of the deal, it just isn’t relevant even though you might argue the valeters have all the information.

The Snapshot System design in the rest of this article can only work if the individual’s medical care is entirely unrelated to the testing. Whatever the person normally does normally for their health continues unchanged. Their GP doesn’t know what the Snapshot service has observed, and the individual doesn’t know either. They are attending a data collection appointment, not a medical appointment.

The reward for the individual comes when the individual’s doctor, through normal clinical channels, gives a medical diagnosis. At that point the doctor may submit clinical evidence to the Snapshot service and request a background report. The service may then release everything in its records relevant to the diagnosis, starting from years back. The individual gains a detailed medical history of the period before their illness was detected, like a medical time machine.

The reward for testing organisation comes: when it has data from a long-term large-scale study that includes a very large number of variables with high frequency and precision. This can be sold, for example to governments who want to build relevant and cheaper health systems, or health companies who would love to have access to this data a few months before most other people see it.

Summary from a medical science point of view: Reducing medical ignorance rapidly requires studies which are difficult, and considered unethical in most jurisdictions. However if we are entirely outside the current medical system and if we avoid all promises or even hints that this is a healthcare service (which is the point of the car valeting analogyt), then the ethical problems are reduced.

Why go to all this trouble?

Medicine has a structural problem: the act of testing someone changes what you’re observing.

Many medical problems including the Snapshot Service come down to the timing and direction of information flows. Usually we expect the timing for medical testing to be as soon as possible, with the exception of followups and a few very specific types of preventative screenings. The normal direction of information flow has been towards the patient, with some flows towards ongoing research, typically with consent.

With the Snapshot System things are very different:

Imagine a research programme is monitoring a group of a few thousand people, and one of the participants is discovered to have a particularly high cholesterol level. Clinical duty requires follow-up and often treatment, so the participant enters the healthcare system and their disease trajectory is altered. The researchers are now observing disease that has been interrupted by medical care, which ejects that person from the observations and also makes the researchers part of the healthcare system. Meanwhile, participants expect that the researchers will let them know if anything is wrong, which conflicts with the researchers’ need to know what happens if there is no intervention.

By explicitly remaining outside the healthcare system, the Snapshot service inverts the timing. Everything flows in to the testing centre during observation. Information flows out only when diagnosis has already occurred through independent channels.

The causality problem in detail

Here is a chain of cause-and-effect events

• Many medical conditions are evident in test results long before they become a problem
• Early detection and treatment significantly improve medical outcomes, when the detection is correct rather than false alarm or other misdiagnosis
• Evidence of long-term medical conditions are frequently found in clinical autopsies or CT scans - such as small cancers - that would certainly have triggered alerts and maybe intervention had tests been taken. With the benefit of hindsight we can say testing would have exposed the patient to unnecessary risk.

The the ready availability of tests can lead to a skewing of outcomes:

• patients can become overly concerned with tests and the results of tests
• doctors and patients can come to depend on tests, or even just the fact that tests are in train, at the expense of other diagnostic or treatment paths
• patients can be subject to unnecessary medical procedures following testing, either due to their own anxiety or the anxiety of doctors
• doctors’ anxiety may related to risk factors for the doctor rather than the patient (worries about promotion, getting sued, etc…) and this can influence testing behaviour
• medical systems can be distorted by the charging regimes for tests to either order too many tests or to order non-optimal tests (doctors making money by signing test forms…)
• patients can forego tests because of the cost, which can impact the patient’s perception as well as possibly be a lack in the medical knowledge relating to that patient

The patient’s mental state and expectations can have a significant impact on medical outcomes and even causation. Setting aside discredited notions of ‘positive thinking’, this means:

• Patients have expectations about knowledge that will be gained from tests
• Patients have reactions to the actual knowledge gained, or their perception of the knowledge gained
• Both mental state and expectations can be greatly affected by mere fact medical testing is taking place

This all leads to the conclusion:

The fact of taking a test (or, more broadly, ‘being observed’) no matter how everyday or simple, has potential side affects unrelated to the nature of the test.

This applies to things like lifestyle interviews too, it’s all medical data. And as I learned from Epidemiology and One Health, the concept of sentinel species applies to humans, so that basic vetinary records about a pet dog or cat can indicate human health problems decades before they are otherwise visible in the owner. The potential for upset and distress in all these situations is quite high, unless the wall between the observer system and the health system is rigorously maintained and understood from the beginning. And that is where the carwash analogy is a helpful reminder about the separation.

The following two statements are self-evident, and somewhat contradictory:

• Many medical conditions can be better treated, or treated with more confidence, if more is known about their history in as much detail as possible: when were the first signs visible (e.g. change in white cell count); did these signs come and go; what was the medical state when symptoms first noticed; etc.
• Regardless of grey areas about when testing should start, regular, broad-spectrum testing is not recommended for most people Even though this is the only way to have better medical history when you need it.

About errors and biases

There are a lot of challenges involved in testing that an observer-only system avoids or can correct over time. Health systems have to make care decisions while coping with facts such as these:

• All testing has an error rate
• All testing has a bias, because we have chosen who and what to test. What we do not test is a choice too
• Not all testing is equal: a large pathology lab differs from a small corner surgery, and tissue takes time to transport, etc.
• Tests can have four outcomes: positive, negative, inconclusive (no safe conclusion can be drawn), and invalid (e.g. problem with data)
• The false negatives and false positives in the previous point are common, and are often guarded against
• However, results such as ‘inconclusive’ and ‘invalid’ can also be false, and are often not reported even though recorded
• All treatment has an error rate
• Some of the easiest treatment error rates to detect are very high (e.g. drug dosages, amputations) and there is evidence to suggest treatment error rates that are not so easy to detect are also high
• Humans find it difficult to intuitively understand error rates, both patients and some clinicians

Complexity problems in medicine

• Medicine of all kinds and at all levels is beset by complexity. Inherent complexity means our total knowledge is a small proportion of the amount to be known, and the positive feedback/network effects associated with growth of knowledge often mean the complexity becomes greater (e.g. Epigenetics.) There is also contextual complexity, to do with the impact of government, ethical and financial factors.

• To demonstrate how quickly complexity builds, consider multi-morbidity. Even some of the best-understood and most treatable diseases are not at all well-understood in the context of multi-morbidity, and beyond the limits of evidence-based medical data. Multi morbidity is increasingly common, and yet we don’t have a good understanding of how treatments should cope with it. Think of how many elderly patients have drugs for multiple conditions, and think of the number of combinations, and that trials involving drugs in combination are almost non-existent.

• Both the environmental and inherent nature of medicine is changing rapidly. The following is a sample selection of things that are commonly thought to differ between successive human generations in modern society: microflora, genetic diversity, background levels of artificial chemicals (body burden), and hormone patterns. Even with just these factors there is great complexity, and if these factors are all changing by generation, the complexity is vastly increased again. Play with permutations and look at some indicative numbers!!!• Evidence-based medicine is often helpful, but is still very incomplete. It is also regarded as the best we currently have.

• Evidence-based medicine is very difficult to put into practice because of human factors in the medical profession worldwide, even at the level of the simplest routine tasks (classic example, intravenous line cleaning checklists, which when applied reduce an entire class of hospital-caused infections to close to zero but which are notoriously difficult to enforce. Or even getting doctors to wash their hands).

• Empirical data on many variables for large sample sizes is rare (even with just one sampling for each individual in the population)

• Empirical data on many variables over time for an individual medical case is rare

• Multi-variable empirical data collected over a sufficient time, when its collection has not skewed outcomes, and when properly interpreted, will always improve medical outcomes

No wonder then that longitudinal studies are regarded as critically urgent by government and also the golden standard by medical companies of all kinds. These studies are regarded as expensive, and worst of all they require long-term commitment by political and medical entities geared to short-term decision cycles.

Ignorance problems in medicine

Intuitively, what is not known about human health is vastly more than the knowledge, yet attention is invariably focussed on the relatively small amount of knowledge we have. I find it helpful to think about medical ignorance rather than medical knowledge, although they are dealing with the same subject.

From this perspective, I suggest:

• It is very expensive to reduce our medical ignorance, and frequently the expense factors have to do with personal, social and regulatory issues rather than medical science.
• Medical ignorance is best addressed by considering data about many people gathered over a long period of time.
• Medical ignorance decreases as the number of valid data collection regimes increases, provided sample sets are large and collected over a long period of time .
• Assuming sound design, the value of a data collection increases with the time it has been running. In other words, the most valuable studies are likely those that have not yet finished.

The early detection problem

When an individual has a disease or health anomaly we want to notice early and treat with maximum intelligence however there are many issues connected with early detection.

There are some classical dilemmas:

• ethical dilemma - we must choose between treating while lacking good knowledge, and delaying treatment until we have good knowledge, with very little middle ground. Each choice can lead to bad outcomes, and they are mutually preclusive.
• medical dilemma - acting prematurely on knowledge about the patient can lead to: under or over-treatment; uncertainty (ie we know there is a very high error margin); entirely wrong results (eg false negative by misinterpreting tests, or wrong limb); and false certainty (ie we don’t know that there is a very high error margin.)
• financial dilemma - inefficiency is expensive. A healthcare system that over or under-tests is inefficient, as is one which generates extra work due to the fallout from errors and uncertainty or false certainty.

Here is an example of the causality paradox: A patient presents with a previously unreported problem, and a doctor decides it should be looked at closely. At that instant, the best medical outcome will arise if a long and very detailed, multidimensional history of the patient is already available.

However, such a history cannot exist within the bounds of today’s medical framework, because, in order to gather such a history:

• we would never get to that instant of decision with little prior background. Given partial previous information we would have been forced to consider alternatives long before
• the process of considering alternatives due to increased (but partial) information frequently leads to interventions being made even though more information could mean no intervention would ever be made
• we may never get to a position of good information because ethically once we’ve started a series of tests we need to intervene once evidence gets to a certain point

That’s the problem in a nutshell - how can we ensure appropriate detailed historical records are available to treat a patient after diagnosis, knowing that if we attempt to make these records beforehand the patient may be worse off?

This is why the Snapshot System is potentially a direct benefit to the individuals.

The lack of evidence problem

This is the second, larger-scale problem addressed by this proposal:

Medical science is ignorant in many fields because good empirical evidence about populations either has not been collected or there are ethical and practical barriers to doing so

Ethical, Medical and Financial Dilemmas:

• ethical dilemma - quality empirical evidence is all about hands-off observation, however, observing a sick patient closely will often yield information that could help the patient.
• medical dilemma - medical action is often required even when the knowledge behind the action is based on incomplete or non-existent empirical data. This is when doctors have to make the best judgement they can, often after careful consultation with collegues but still essentially a guess. Part of the skill of being a doctor is handling two areas of poor information at once: that of an individual’s history and that of the science bearing on the individual case.
• financial dilemma - good empirical datasets are very expensive to gather, and take a lon

As a recent example, I found when studying of malaria, seemingly obvious facts such as who has natural immunity or the circumstances of desperately ill children are simply unknown. Similarly, my work in ventilation for health is almost unique for collecting indoor air quality data, despite this being where most people spend most of their lives, and where some of the biggest problems are found 🤷‍♀️.

The Snapshot Service is not a new idea

The US Department of Defense Serum Repository, established in 1985, holds over 55 million serum specimens from more than 10 million military personnel. Samples are collected at enlistment and at intervals, without individual feedback during service. The repository releases stored specimens to military clinicians when illness is diagnosed, which is the Snapshot’s retroactive release feature.

China’s “Physicals for All” programme (Tibet from 2013, Xinjiang from 2016) involved annual health examinations of tens of millions of people, with biometric data collected and no individual results returned. There is detailed debate within Chinese scholarship of the legal and ethical challenges of these systems and, similarly, with the US Defence Department system. Policy and law have changed over the years but both systems function as they were initially designed.

This does show that population-scale testing without individual feedback is administratively feasible.

Newborn dried blood spot programmes, operating in most countries since the 1960s, have been used to retroactively confirm congenital infection as the cause of childhood hearing loss, years after the original collection. This type of application seems to be quite readily accepted by participants and ethecists worldwide.

The aspects I am interested in are beyond the basic mechanics. Can we combine voluntary consent with personal clinical benefit? Does it make sense to apply modern cryptographic techniques to protect against state-mandated access to the data? I have an interest in crytogaphically time-locked data, can this apply?

A comparison of 26 health observation systems covers every comparable programme I could find across nine decades, scored against 18 features, with source criticism and bias analysis for each. Much of the existing commentary on health observation systems carries strong assumptions about what counts as ethical, and those assumptions vary by tradition. The comparison addresses this directly.

Each of the 26 chose differently, and each is incompatible with what the Snapshot needs.

UK Biobank as a local case

UK Biobank is the largest voluntary population biobank ever assembled. and explicitly prohibits individual feedback. The assessment visit “is not a health check” and participants “will not receive personal results.” This no-feedback policy has seemingly been maintained against substantial pressure, and around 500,000 participants accepted it. The financial model appears to be successful too.

UK Biobank does not have a mechanism for releasing individual-level pre-diagnostic data to a participant’s treating doctors when illness is diagnosed. It recruited a single cohort over four years. It operates under UK law and acknowledges it will resist court orders but cannot guarantee success.

What Remains Hard

The first problem is the case where the testing system detects something the normal medical system is missing. The only option consistent with the service’s function is to remain silent. Any exception for “the medical care is clearly wrong” creates a pathway that will expand. This is a large part of what I am trying to address.

The second is infectious disease. The service might detect active tuberculosis long before the normal system does, and mandatory reporting is incompatible with the embargo architecture. Various protocols have been tried by these systems. There is a difference between an observer system interacting with the normal health system, and interacting with epidemiological surveillance systems, and this is worth exploring.

Where Things Stand

The technical components are available: the measurement devices, the data security tools, the legal structures, the commercial interest in better longitudinal data. The governance architecture is the work that remains. Developing it requires people across trust law, data security, clinical research ethics, medical informatics, and commercial health investment.

The full analysis covers all 26 systems, the scoring methodology, and how different legal and ethical traditions — Chinese, African, European, American and others — would change the design.