Skip to main content

In-app reporting

The in-app reporting feature empowers the health worker to see how she is performing against her daily task list, and allows for offline peer-to-peer syncing of the data collected on a daily basis. This greatly simplifies reporting requirements by eliminating the need for cumbersome paper registries and tally sheets.

Measure​

A Measure is a FHIR resource which represents definition- for calculation of an Indicator. The Measure uses a logic Library that contains the calculation logic for measure that is written as a Clinical Quality Language (CQL) expression. The Measure resource represents a structured, computable definition of a health-related measure such as a clinical quality measure, public health indicator, or population analytics measure. A quality measure is a quantitative tool to assess the performance of an individual or organization with respect to a specified process or outcome via the measurement of actions, processes, or outcomes of clinical care. Quality measures are often derived from clinical guidelines and are designed to determine whether the appropriate care has been provided given a set of clinical criteria and an evidence base.

Library - FHIR v4.6.0 resource is referenced that contains the logic required by the measure, and the various expression elements, such as population criteria, reference named expressions within that library (or libraries). In addition, if the Measure references multiple libraries, then any expression references within the resource must be qualified with the name of the library that contains the referenced expression.

The complexity involved in specifying the criteria in the general case requires the use of a high-level query language such as Clinical Quality Language (CQL). As such, the Measure resource defines only the top-level populations and references expressions for the actual criteria. These expressions are typically provided using a Library Library - FHIR v4.6.0 resource containing CQL or ELM expressions. In addition, the individual members of a population may be cases such as encounters or procedures and in these cases, the Group resource would be unable to represent the population characteristics accurately.

MeasureReport​

The MeasureReport resource represents the results of calculating a measure for a specific subject or group of subjects. The $evaluate-measure operation of the Measure resource is defined to return a MeasureReport. The resource is capable of representing three different levels of report: individual, subject-list, and summary.

The Measure resource represents a structured, computable definition of a health-related measure such as a clinical quality measure, public health indicator, or population analytics measure. A quality measure is a quantitative tool to assess the performance of an individual or organization with respect to a specified process or outcome via the measurement of actions, processes, or outcomes of clinical care. Quality measures are often derived from clinical guidelines and are designed to determine whether the appropriate care has been provided given a set of clinical criteria and an evidence base.

Note that the Measure itself does not typically contain any logic; rather a Library resource is referenced that contains the logic required by the measure, and the various expression elements, such as population criteria, reference named expressions within that library (or libraries). This is done using the Clinical Quality Language syntax. The library will contain a CQL-ELM embedded as a base64 . Read more here Clinical Quality Language (CQL)

In addition, if the Measure references multiple libraries, then any expression references within the resource must be qualified with the name of the library that contains the referenced expression.

Read more here Clinicalreasoning-quality-reporting - FHIR v4.6.0 and here HL7.FHIR.US.CQFMEASURES\Glossary - FHIR v4.0.1

Types of MeasureReports​

  1. Cohort A measure score in which a population is identified from the population of all items being counted. For example, one can identify all the patients who have had H1N1 symptoms. This population is very similar to the Initial Population but is called a Cohort Population for public health purposes.

  2. Continuous Variable A measure score in which each individual value for the measure can fall anywhere along a continuous scale and can be aggregated using a variety of methods such as the calculation of a mean or median (for example, mean number of minutes between presentation of chest pain to the time of administration of thrombolytics)

  3. Proportion A score derived by dividing the number of cases that meet a criterion for quality (the numerator) by the number of eligible cases within a given time frame (the denominator) where the numerator cases are a subset of the denominator cases (for example, percentage of eligible women with a mammogram performed in the last year).

  4. Ratio A ratio is a score that is derived by dividing a count of one type of data by a count of another type of data. For example, the number of patients with central lines who develop infection divided by the number of central line days

A working example (Households and Members disaggregated by age) of Measure can be found here. Notable components in example Measure are

{
  "url": "http://fhir.org/guides/who/anc-cds/Measure/HOUSEHOLDIND01",
  "name": "HOUSEHOLDIND01",
  "relatedArtifact": [ {
    "type": "depends-on",
    "label": "FHIRHelpers|4.0.1",
    "resource": "Library/1753"
   } ],
  "library": [ "http://fhir.org/guides/cqf/common/Library/HOUSEHOLDIND01" ],
  "scoring": { ... },
  "group": [ {
    "id": "males",
    "population": [ {
      "id": "initial-population",
      "code": { ... },
      "criteria": {
        "language": "text/cql-identifier",
        "expression": "Patients"
      }
    }, {
      "id": "denominator",
      "code": { ... },
      "criteria": {
        "language": "text/cql-identifier",
        "expression": "Group Member"
      }
    }, {
      "id": "numerator",
      "code": { ... },
      "criteria": {
        "language": "text/cql-identifier",
        "expression": "Males"
      }
    } ],
      "stratifier": [ {
        "id": "by-age",
        "criteria": {
            "language": "text/cql-identifier",
            "expression": "Age Stratifier"
        }
      },
      { ... ... ... } ]
  },
  { ... ... ... } ],
  "supplementalData": [
      {
          "description": "groups",
          "criteria": {
              "language": "text/cql-identifier",
              "expression": "Group"
          }
      }
  ]
}

Details of notable fields for Measure

  1. url: The complete url of Measure with Measure.name i.e. http://fhir.org/guides/who/anc-cds/Measure/HOUSEHOLDIND01

  2. name: A unique name for Measure i.e. HOUSEHOLDIND01. The Measure is loaded by name and url into measure processor

  3. relatedArtifact: Helper libraries to load before running Measure to be used by measure logic libraries i.e. FHIRHelpers|4.0.1. The Library is loaded by its canonical url i.e. Library/123

  4. library: The CQL logic libraries used by Measure for calculation i.e. http://fhir.org/guides/cqf/common/Library/HOUSEHOLDIND01. The logic library is loaded by its url. The url must end with /Library/name

  5. scoring: proportion | ratio | continuous-variable | cohort

  6. group: The section of report defining the stats for a specific indicator (may be disaggregated by stratifier). Each group has following components

    • id: Group name/id.
    • population: The calculations (or name of variable in CQL that defines the value) for each of population components i.e. initial-population, denominator, numerator
    • stratifier: The disaggregations for given population numerator. i.e. by age, by month, by education etc
    • supplementalData: Any extra data or intermediate calculation to be output to final report. Current implementation of MeasureEvaluator does not allow running any measure which is not Patient centric i.e. a Measure.subject can always be a Patient. Hence, we are using supplementalData to output Group for each indicator and then counting distinct Group to count Households

CQL Logic/Decision Library for Measure​

The Measure needs a Library to get the logic/calculation for ecah variable. This calculation or logic comes from CQL. Here is a detailed guide on CQL. Some example CQL scripts can be found here.

A working example of used by above Measure is here. Some notable lines are

library HOUSEHOLDIND01 version '1'
...
include "FHIRHelpers" version '4.0.1' called FHIRHelpers
...
// The Measure interval has closed boundaries. Read details https://cql.hl7.org/02-authorsguide.html#interval-values
parameter "Measurement Period" Interval<DateTime>
context Patient
...
define "All Groups": [Group] G where G.type = 'person'
define "All Group Members": flatten("All Groups" G return (G.member M return M.entity))
define "Group": "All Groups" G return G.id
define "Group Member": "All Group Members" G where "Patient Id" = Split(G.reference, '/')[1]
define "Patients": {Patient}

define "Males": Patient.gender='male'
define "Females": Patient.gender='female'

define "Age": CalculateAgeInYearsAt(Patient.birthDate, ToDate("Measurement Period".high))

define "Age Stratifier":
  case
    when "Age" < 1 then 'P0Y'
    ....
  end

The CQL is referenced by url into Measure. The CQL is translated into elm-json and uploaded a Library on server. There are multiple ways to get elm for CQL. (elm-xml is also a valid standard but it is not recognized by android fhir libraries yet)

CQL to ELM REST Translator

A elm REST app that can be used to run elm microservice and convert CQL via a REST API.

CQL to ELM JAVA Translator

A elm java app that can be used to elm translator on files and get an output. Instructions can be found here

Note : Above approaches output a json elm which then need to be base64 decoded and copied to the Library content as Attachment.

How to Test your CQL Script

The fhir-resources repository has a testing module which allows to not only get the complete Library resource to directly save to server but also allows to test the Measure output and make changes on the fly. Check the cucumber tests Feature File, the Test Code File and the Convertor Util Method

OpenSRP Unit Tests

The CQL can also be translated to Library using an approach as used by OpenSRP as in CQL Content Tests. A complete Library resource is output to console as a result.

Testing the Measure CQL Library​

To make sure your Measure and Library are working and have been validated data, it is best to thoroughly test the input and output first so that multiple updates to server can be avoided and an easy and quick test driven approach is opted to implement your new functionality. The fhir-resources repository has a testing module which implements Cucumber tests to help facilitate this.

  • Checkout fhir-resources Repository
  • Open module fhircore-testing into Intellij or VS Code
  • Install Cucumber plugin (optional and allows to run feature file directly)
  • The module has a lot of helper functions to allow creating sample data and testing basic MeasureReport output. if you understand Cucumber you can add a complete custom test as well
  • Add you test feature file to fhircore-testing/src/test/resources/measure-report/ i.e. fhircore-testing/src/test/resources/measure-report/household-members.feature
Feature: Household Members Count by Age group

  Scenario: Household members in household disaggregated by age group
    Given Household Members CQL is "ecbis/measure_cql/household_measure_reporting.cql"
    Given Household Members Measure is "ecbis/measure/household_measure.fhir.json"
    Given Household Members Sample Bundle has 19 Groups; of them 17 active person
    Given Household Members Sample Bundle has 16 Group Members gender "male" active aged
      | 0 | 1 | 4 | 5 | 6 | 9 | 14 | 15 | 16 | 25 | 34 | 40 | 49 | 50 | 51 | 55 |
    Given Household Members Sample Bundle has 21 Group Members gender "female" active aged
      | 0 | 1 | 2 | 4 | 5 | 6 | 7 | 9 | 14 | 15 | 16 | 21 | 25 | 34 | 40 | 44 | 48 | 49 | 50 | 51 | 55 |
    When Household Members Measure is run
    Then Household Members Measure Report has "evaluatedResource.count()" = "17"
    Then Household Members Measure Report has "group.count()" = "2"
    Then Household Members Measure Report has "group.where(id='males').population[0].count" = "37"
    Then Household Members Measure Report has "group.where(id='males').population[1].count" = "37"
    Then Household Members Measure Report has "group.where(id='males').population[2].count" = "16"
    Then Household Members Measure Report has "group.where(id='females').population[0].count" = "37"
    Then Household Members Measure Report has "group.where(id='females').population[1].count" = "37"
    Then Household Members Measure Report has "group.where(id='females').population[2].count" = "21"
  • 'Household Members' is the TAG. This should match your Measure core functionality
  • 'ecbis/measure_cql/household_measure_reporting.cql' is cql path. This should be the path from main dir i.e. fhir-resources where your cql resides
  • 'ecbis/measure/household_measure.fhir.json' is measure path. This should be the path from main dir i.e. fhir-resources where your measure resides
  • Change test data as per your requirements. You can change values into double quotes, int values, or in data tables as per your test requirements
  • Make sure to use same Step definition convention i.e. Given TAG CQL is "cql-path" OR Given TAG Measure is "measure path". Otherwise you would need to write your own Steps into YourTagTest.kt file using standard defined in Cucumber Tutorial
  • The assertions should also use same Step convention. i.e. Then TAG Measure Report has "your-fhirpath-in-measure-report" = "expected-size". Or you can add additional assertions to Test code file as in Then section below
  • Add your test file to kotlin/com/fhircore/resources/testing/measure/YourMeasureNameTest.kt i.e. kotlin/com/fhircore/resources/testing/measure/HouseholdMembersMeasureTest.kt

Note The feature-file name should match with test-file name i.e. household-members.feature corresponds to HouseholdMembersMeasureTest.kt

@RunWith(Cucumber::class)
@CucumberOptions(
  features = ["src/test/resources/measure-report/household-members.feature"], tags = "not @ignored"
)
class HouseholdMembersMeasureTest : En {
  private val measureContext = MeasureContext().apply { withMeasurePeriod("2022-01-01", "2022-07-15") }

  private val TAG = "Household Members"

  init {
    measureContext.givenCql(TAG) // helper step definition to load and print Library in console
    measureContext.givenMeasure(TAG) // helper step definition to load and print Measure in console

    // add your test data below. you can add custom code as well to load data of your choice
    measureContext.givenGroupsXHavingYActivePerson(TAG) // helper step definition to add households test data
    measureContext.givenGroupMembersXHavingGenderYAndAgesZ(TAG) // helper step definition to add household members, patients test data

    // helper step definition to run measure of given type and print report in console
    measureContext.whenMeasureRun(TAG, MeasureEvalType.POPULATION)

    // helper step definition to add assertions. You can add custom assertions as well to test complex data
    Then("Household Members Measure Report has {string} = {string}") { path: String, value: String ->
      measureContext.thenMeasureReportHas(path, value)

      // Add additional assertions if needed
      // assertEquals(0.5, measureContext.measureReport.group.first().measureScore)
    }
  }
}

The final working Library, Measure, and MeasureReport are printed to console which can be copied and POST/PUT to server.

$evaluate-measure operation​

The $evaluate-measure operation is used to calculate an eMeasure and obtain the results

Parameters

UseNameCardinalityTypeDocumentation
INperiodStart1..1dateThe start of the measurement period. In keeping with the semantics of the date parameter used in the FHIR search operation, the period will start at the beginning of the period implied by the supplied timestamp. E.g. a value of 2014 would set the period start to be 2014-01-01T00:00:00 inclusive
INperiodEnd1..1dateThe end of the measurement period. The period will end at the end of the period implied by the supplied timestamp. E.g. a value of 2014 would set the period end to be 2014-12-31T23:59:59 inclusive
INmeasure0..1string (reference)The measure to evaluate. This parameter is only required when the operation is invoked on the resource type, it is not used when invoking the operation on a Measure instance
INreportType0..1codeThe type of measure report: subject, subject-list, or population. If not specified, a default value of subject will be used if the subject parameter is supplied, otherwise, population will be used
INsubject0..1string(reference)Subject for which the measure will be calculated. If not specified, the measure will be calculated for all subjects that meet the requirements of the measure. If specified, the measure will only be calculated for the referenced subject(s)
INpractitioner0..1string (reference)Practitioner for which the measure will be calculated. If specified, the measure will be calculated only for subjects that have a primary relationship to the identified practitioner
INlastReceivedOn0..1dateTimeThe date the results of this measure were last received. This parameter is only valid for patient-level reports and is used to indicate when the last time a result for this patient was received. This information can be used to limit the set of resources returned for a patient-level report
OUTreturn1..1MeasureReportThe results of the measure calculation. See the MeasureReport resource for a complete description of the output of this operation. Note that implementations may choose to return a MeasureReport with a status of pending to indicate that the report is still being generated. In this case, the client can use a polling method to continually request the MeasureReport until the status is updated to complete

The effect of invoking this operation is to calculate the measure for the given subject, or all subjects if no subject is supplied, and return the results as a MeasureReport resource of the appropriate type. Note that whether or not this operation affects the state of the server depends on whether the server persists the generated MeasureReport. If the MeasureReport is not persisted, this operation can be invoked with GET

The MeasureReport output​

The resulting MeasureReport of above Measure is here. Some important components of report are below

{
  "resourceType": "MeasureReport",
  "contained": [
    {
      "resourceType": "Observation", ...
      "extension": [ {
          "url": "http://hl7.org/fhir/StructureDefinition/cqf-measureInfo",
          "extension": [...., {
              "url": "populationId",
              "valueString": "group"
           } ]
      } ],
      "code": {
        "coding": [ {
            "code": "Group/1818d503-7226-45cb-9ac7-8c8609dd37c0/_history/3"
        } ]
      }, ...
  } ],
  "type": "summary",
  "measure": "http://fhir.org/guides/who/anc-cds/Measure/HOUSEHOLDIND01",
  "date": "2022-06-28T12:28:28+05:00",
  "period": {
    "start": "2022-01-01T00:00:00+05:00",
    "end": "2022-06-28T23:59:59+05:00"
  },
  "group": [ {
      "id": "males",
      "population": [ {
          "id": "initial-population",
          "code": { ... },
          "count": 136
        }, {
          "id": "denominator",
          "code": { ... },
          "count": 7
        }, {
          "id": "numerator",
          "code": { ... },
          "count": 3
      } ],
      "measureScore": { "value": 0.42857142857142855 },
      "stratifier": [ {
          "id": "by-age",
          "stratum": [ {
              "value": { "text": "P0Y" },
              "population": [ {
                  "id": "initial-population",
                  "code": { ... },
                  "count": 31
                }, {
                  "id": "denominator",
                  "code": { ... },
                  "count": 4
                }, {
                  "id": "numerator",
                  "code": { ... },
                  "count": 2
              } ],
              "measureScore": { "value": 0.5 }
            }, { ... ... ... ...}
              ]
            }
          ]
        }
      ]
    }
  ]
}

Notable components of the MeasureReport

Details of some notable fields in above MeasureReport (calculated from Measure) are

  1. contained: The Measure property supplementalData is calculated for each measure subject and output as an Observation having extension http://hl7.org/fhir/StructureDefinition/cqf-measureInfo with inner extension defining the variable requested i.e. group in case above. The code.coding.code has the value of given variable Group/1818d503-7226-45cb-9ac7-8c8609dd37c0/_history/3 in example above

  2. measure: The Measure.url for which this report was generated i.e. http://fhir.org/guides/who/anc-cds/Measure/HOUSEHOLDIND01

  3. period: Measure period which was sent for date filter i.e. reporting period start and end. The Measure interval has closed boundaries. Read details here

  4. group: The calculated value for each Measure indicator with

    • id: id/name of group/indicator/stratifier as defined in Meaure.
    • count: Calculated value from CQL for given variable
    • measureScore: The percent/ratio of calculated value i.e. numerator/denomintor. Note that for stratifier the score denominator is stratifier denomintor rather than group denominator

  5. stratifier: Count for given indicator disaggregated by each type. The stratifier misses the values where counts are zero. Hence if stratifier has predefined criteria, each should be a calculated as separate group.

OpenSRP Integration​

Once your Measure, and Library is ready add these to sync_config.json to make sure that the Measure and all dependent Library resources are always synced.

  1. Save/Update Measure on server i.e. POST - https://your.fhir.server/fhir/Measure OR PUT - https://your.fhir.server/fhir/Measure/measure-id
  2. Save/Update Library on server i.e. POST - https://your.fhir.server/fhir/Library OR PUT - https://your.fhir.server/fhir/Library/library-id
  3. Update the sync_config.json Debug or Binary config for your app with new your measure id, and library id as below
  {
    "resource": {
      "resourceType": "SearchParameter",
      "name": "_id",
      "code": "_id",
      "base": [
        "Measure"
      ]
    },
    "type": "token",
    "expression": "133082,133104,{your-measure-id}"
  },
  ... ... ...
  {
    "resource": {
      "resourceType": "SearchParameter",
      "name": "_id",
      "code": "_id",
      "base": [
        "Library"
      ]
    },
    "type": "token",
    "expression": "1753,133081,133105,{your-library-id},{any-helper-library-ids}"
  }
  1. Update the measure_report_config.json Debug or Binary config with your new measure so that it shows up in the list
"reports": [
  ... ... ...
    {
      "id": "new serial id in list",
      "title": "Household Members",
      "description": "Number of Households, Household members registered, disaggregated by age and gender for each age group",
      "url": "http://fhir.org/guides/who/anc-cds/Measure/HOUSEHOLDIND01"
    },
    ... ... ...
  ]

Screenshot of MeasureReport on OpenSRP​