Overview

August 29, 2021

Complexity, don’t walk away from it….. (but RUN!!!)

Complex Data Model

About 5 years ago I visited, in function of ICT-consultant, a hospital. This was an academic hospital, a very large one. It had a wide range of services and treatments, and was always number one in innovating. It was known as the best hospital in that country.

Around 1995, the hospital decided to build a hospital-wide medical information system, also this was very innovative. And they kept change in mind, because they knew, nothing changes as fast as scientific data-structures, and especially this is the case for medical data, which are not only subject of changes in science, but also changes in law and society.

On the GUI-side, they had a smart idea how to give users easy and quick access to functionality. They had a system of menus which remembered the menu-items a specific user uses. So a user would only see a few number of menu-items, while in fact, the system could grow to a large number of menu-items. And it did.

The best information-architects at work, very bright people, but they overlooked something very important. It is a fact that normally talented developers can comprehend 150 database-tables, how they interact, which data they contain, keys, relations, normalization. The very bright people may perhaps double that number, but I doubt it.

But they started defining the system with the needs from the moment, which resulted in 1000 database-tables as the application went alive. That was already too many, but it was only starting.

The system exploded.

Almost every day new inventions, new policies, new laws, society changes were to implement. Because no-one was able to understand the meaning of the database-tables anymore, developers started to create new ones for every new function. At the time I looked, the system was 20 years running, it had 10.000 tables. So it grew with 500 per years, that was 2 every working day. Querying a simple thing like blood-pressure-history was impossible. Blood-pressure could exist on many location in the database, and the context was very hard to understand or unknown. Even the unit used when a data-item was stored was not always clear.

The semantic meaning of the data was hidden in the windows that were used to enter them, and those windows were hidden under menu-items which were not visible, only to them who used them regularly. The table-names and field-names did not added much. Often they were technically oriented abbreviations.

So even if you could find a blood-pressure in a table, was it possible to use it in a historic blood-pressure overview for a patient? Maybe it was the blood-pressure during an operation, or during a medical treatment with temporarily effect on the blood-pressure. It required medical education and lots of time for a clinician to find its meaning, for a developer impossible.

So the hospital had lots of data, but the data were useless.

Purpose

  • Separate code from data: Data are manipulated using generic general purpose functions, having polymorphism without objects.
  • Low code in clients: Data reflect the domain structures directly and enable working with low code and RAD in connecting applications.
  • Documentation of data: Data are implicitly documented via archetypes and reference models.
  • Validation: All data are validated against archetypes, the archetypes are validated against the reference models.
  • Flexibility: Data-structures can change, but data remain documented and retrievable, also in previous structures.
  • Domain driven: The reference models, templates and archetypes are able to reflect any domain in full complexity and versioning. Domain professionals directly define the content and other semantics of implemented solutions rather than IT developers.

Background

Around the year 2000, Thomas Beale e.o. started the OpenEhr-project. In fact this is a Semantic Database, but with one reference model and one purpose. I got involved around 2005, and from the beginning I thought of having support for more reference models simultaneously would enclose the potentials of the basics of the project. In the beginning I started implementing additional reference models, hard coded, such as EN13606, and I started creating interoperability layers to other data-models such as HL7-models.

Goal

In this system I want to separate code and reference-models, separate data from code, and in this way supporting virtually any domain without any performance penalty. Having the basic semantic-enabling structures of the basic OpenEhr-system, all data remain semantically explained and are flexible in adapting new requirements.

Another purpose is to divide the system in modules/micro-services which will run independently and communicate over gRPC and REST. In this way, the system can run on one system, or more, or on cloud-services.

Running the public interfaces on REST, makes the system programming language and methodology agnostic.

The semantic database will also be able to run an Openehr Reference Model in any version and use the according archetypes..

Interacting modules/services

August 29, 2021

Process of creating a Reference Model

The Reference Model is the Domain Model

The information stored in the EMF models can be used to generate derived client-output. A typical use case is that you use EMF to define the domain model of your application and that you generate the corresponding implementation classes from this model.

The Reference Model Module is a background support system which can support a GUI-Reference-Model-designer, or work with text-definitions like OCL or XSD or descriptive in JSON or XML. Every time when needed, the storage module can store an incomplete/unfinished reference model, and when finalizing, the reference model becomes immutable and the name/version combination will not be available for another reference model.

Process of creating an Archetype

Archetypes are the building-blocks of the information-system. They define and explain constraints on the classes in the Reference Model. For example, if a Reference Model would have a property which would be a list of items. The archetype could define/restrict the number of items, and/or specify which kind of items. An archetype is a specification of a Reference Model class. Besides that, an Archetype contains explanation about how and why, copyright, other metadata. And an archetype has an ontology-section which explains every property.

The Archetype Module is a background support system which can support a GUI-Archetype-designer, or work with text-definitions like ADL, JSON or XML. Every time when needed an Archetype can be validated against the Reference Model using the reference Model Module. Every time when needed, the storage module can store an incomplete/unfinished archetype, and when finalizing, the archetype becomes immutable and the name/version combination will not be available for another archetype.

Reference Model

August 29, 2021

Reference Model Module

The Reference Model is the Domain Model

The Reference Model Module will be build around EMF and client-code can be generated using Acceleo

The core EMF framework includes a meta model (Ecore) for describing models and runtime support for the models including change notification, persistence support with default XMI serialization, and a very efficient reflective API for manipulating EMF objects generically.

The Meta-model for EMF looks like this:

We use this model and Java-library to distill following API from it:

Acceleo is the result of several man-years of R&D. MTL is an implementation of the Object Management Group (OMG) MOF Model to Text Language (MTL) standard. The OMG MTL standard offers outstanding advantages : High ability to customize, Interoperability, Easy kick off, and more!

Acceleo is a template-based technology including authoring tools to create custom code generators. It allows you to automatically produce any kind of source code from any data source available in EMF format.

Acceleo - Overview

Acceleo is based on three separate components: the compiler, the generation engine and the tooling. The acceleo runtime is composed of the generation engine and the compiler.

Compiler

The Acceleo compiler can be found in the bundle org.eclipse.acceleo.parser. In this bundle, you can use the class AcceleoParser to parse a “mtl” file and return the root of the “emtl” file. That root can then be serialized in a file with the “emtl” extension. The Acceleo parser can parse Java.io.File, org.eclipse.core.resources.IFile or java.lang.StringBuffer and then create the emtl file matching the mtl File or IFile or it can return the emtl in EMF resource.

Generation engine

The generation engine can launch an Acceleo generation from an Acceleo module and and EMF model.

Standalone

The Acceleo runtime can be used in a stand alone environment without any dependencies to the Eclipse IDE, as such it can be embedded in a good old Java application. Acceleo 3.1 will feature some specific Ant and Maven tasks in order to call the Acceleo compiler and the Acceleo generator in stand alone easily.

MODEL-TO-TEXT

MODEL-TO-TEXT transformation is the process (technique) of extracting code from a model. This tranformation could be defined in several ways, depending on your source metamodel and your destination language. It’s the opposite of TEXT-TO-MODEL, that extracts a model analysing the source code.

For instance, you can extract and generate Java code from a UML in a MODEL-TO-TEXT transformation.

Acceleo, for example, is a tool based on standars which makes you possible to make a tranformation MODEL-TO-TEXT. Acceleo needs model (maybe in .xml format or the standar .xmi) and some templates (.mtl) that you define.

With these templates and the Acceleo engine you will be able to parse your model and generate the code you need.

To sum up: MODEL-TO-TEXT is a process, and Acceleo is a tool to develop and run the process.

Archetype

August 29, 2021

Template

August 29, 2021

Data

August 29, 2021

Storage

August 29, 2021

UCUM

August 29, 2021

Query Engine

August 29, 2021

Reis naar Katja, laatste rustdag

August 20, 2021

Tja, deze laatste dag. Hoe is die verlopen. Ik ben St Emillion ingegaan, te voet, om ervan te genieten. Vanmiddag rond twaalf uur deed ik die wandeling, bergop, maar niet te heftig. Ik had een soort hemels gevoel, dat ik op dit stukje rijke aarde mag rondlopen. Wat is dat heerlijk. Het zonnetje blikkerde aan de blauwe hemel.

Ik liep door de wijngaarden. De chateaux liggen altijd verder van de weg. En dat “vente ici” en “degustation gratuit” klinkt toch als een erg naïeve toeristenval. De toerist komt binnen, wordt ingeschat op wat hij wil uitgeven, wordt in no time halfdronken en koopt dan zes flessen voor een volslagen willekeurige prijs zonder enig idee van de waarde en zonder met zekerheid te weten of erin zit wat hij geproefd heeft.

Met Kerstmis over 5 jaar komt hij erachter, want hem is op het hart gedrukt dat een St Emillion minimaal 5 jaar moet liggen. Maar zelfs dan zal hij zichzelf vertellen dat het best wel een goede wijn is die hij voor een aardig prijsje bij de boer zelf heeft gekocht.

En na drie glazen proef je het toch niet meer.

Wat een sarcastisch cynisme.

Santé, zegt de boer ‘s avonds tegen zijn familie bij het diner.

Er is een gouden regel bij wijn. Goede wijn is altijd duur. Slechte wijn soms ook

Maar de Cabernetjes hangen weer aan de wijnstokken, paarszwart, een kleine druif, naar het schijnt amper eetbaar. Maar de beste voor wijn.

Het schijnt dat de Creedence, ouderen onder ons, bekend vanwege mooi gitaarspel en rauwe stem in Bordeaux heeft gewoond. Ze kwamen er op het idee van “I heard through the grapevine”. Dat je door een rij druiven loopt en in een andere rij een gesprek hoort. Je weet niet wie het zegt, of je wil niet toegeven dat je het wel weet.

En dat was hier.

Na enige tijd net bereikte ik “the village on the hill”

In de kerk was een gratis concert. Het was iets amateuristisch, maar wel mooi. Een mezzosopraan begeleid door kerkorgel.

Een beetje gewandeld door het ketkcomplex, hier en daar Esscheriaanse doorkijkjes in de vroeg gotische stijl. Altijd mooi, dat soort dingen

En tenslotte St Emillion, die hier werkelijk heeft geleefd.

Levens verhaal zelf googelen.

Dit is het einde van mijn blog. Fijn dat jullie deelgenoot waren van 1250 km soms barre, soms mooie tijden. Jullie hebben mij erg geholpen. Daar dank ik jullie allemaal voor.

Dag lieve mensen