FHIR Digest

@Grahame Grieve when you can share any updates, we are about to have a conversation with HL7 CG WG (Tuesday April 4th) on this topic - from a broader perspective of "FHIR Test Cases" in general. Just wondering if any notes can be shared.

not at this point

only that I am planning to add a set of test cases for $expand and $validate-code to the test cases github repo that explore known issues around expansion and validation

@Grahame Grieve , @Michael Lawley : If I may believe this: https://confluence.hl7australia.com/display/COOP/2023-03+Sydney+Connectathon , the Australian connectathon was on March 23rd... Do you already have some news about the Publisher/Terminology server interface?

nothing final yet. I will update when there's news

ok there's news. There's test cases here: https://github.com/FHIR/fhir-test-cases/tree/master/tx

From the next release of the validator, you can run them like this:

java -jar validator.jar -txTests -source https://github.com/FHIR/fhir-test-cases -output /Users/grahamegrieve/temp/txTests -tx http://tx-dev.fhir.org -version 4.0

Where:

• -txTests.- run the tests
• -source - the source of the tests (github URL, or a local folder)
• -output - the local folder where you want output to go
• -tx - the address of the server to test
• -version: the version of FHIR to use (default = 5.0, can be 4.0, 3.0)
• -filter: string portion of a particular test to run

there's a fair bit of work to go here, but this is the shape of where things are going

@Grahame Grieve What's the preferred way to provide feedback on these tests - questions, apparent bugs, etc?

Currently I have issues with the REGEX test, a bunch of the language tests, and the big-echo-no-limit test which seems to require a system to refuse to return an expansion with more than 1000 codes?

Wrt the language tests, language-echo-en-en, language-echo-de-de seem to suggest that the expansion should set ValueSet.language based on the displayLanguage parameter to $expand. But, that would then imply that the entire result ValueSet is in that language rather than just the ValueSet.expansion.contains.display values (which is all that parameter is really requesting).

For the translated CodeSystems in the language tests, none of the translations have a use value, so I (Ontoserver) can't know that they should be used as the preferredForLanguage display value.

Last question: is there a branch available with the -txTests option

What's the preferred way to provide feedback on these tests - questions, apparent bugs, etc?

discussion here first, I think.

Currently I have issues with the REGEX test

what?

the big-echo-no-limit test which seems to require a system to refuse to return an expansion with more than 1000 codes?

well, this is something we'll have to figure out. It's my test that that's how my servers work. It's not necessarily how other systems have to work, so we'll have to figure out how to say that in the tests

Wrt the language tests, language-echo-en-en, language-echo-de-de seem to suggest that the expansion should set ValueSet.language based on the displayLanguage parameter to $expand. But, that would then imply that the entire result ValueSet is in that language rather than just the ValueSet.expansion.contains.display values (which is all that parameter is really requesting).

I sure expected some discussion on this. There's two different things that you might want - languages on display, and languages on the response. The way the tests work, if you specify one or more display languages, you get displays defined for those languages

But the language of the response - the ValueSet.language, that's based on the language parameter of the parameters, which the controls how the available displays are represented in the response, based on the value of ValueSet.language

with regard to the use parameter, I don't believe that the spec says anywhere that there is a preferredForLanguage code, so how can that be in the tests?

is there a branch available with the -txTests option

the master has that now

I now have the validator test runner going, but I think it is being really overzealous in the level of alignment its looking for between the expected response and the actual response.
First two issues: .meta and .id -- I don't think either of these should be included in the comparison.
Next one: ValueSet.expansion.id -- that's purely a server-specific value

.meta and .id... I'm not producing them, right?

.expansion.id? or expansion.identifier?

Regarding the regex issue, we're limited to Lucene's flavour which does not include character classes like '\S' or '\d'.

.id is in simple/simple-expand-all-response-valueSet.json for example. I produce .meta but not .id

ouch. would you like to propose an alternative regex?

".{4}[0-9]" would work for me in this example, but it's not quite the same. The more accurate "[^ \t\r\n\f]{4}[0-9]" would also work.

And yes, I did mean expansion.identifier, but I think this was a false negative -- me misreading the output

I will commit some changes when I can

• replace the regex
• ignore meta
• require no id

btw, what are you putting in meta?

Require? no id or just don't care? I think a bunch of things should be don't care

Meta was including a version (doesn't really make sense) but also a lastUpdated

I think for id, it shouldn't have an id? I just stopped regurgitating the id, which was basically an oversight

It would also potentially propagate tags

What if it's using a stored expansion?

in this context?

Well, no, but I'm thinking that these tests should really only be looking for things that are known to be wrong

perhaps. They're also my own internal qa tests. that might be too much, I guess, but I'm hoping not

I was thinking that the expected response in the test would set the scope of required elements, and other things would just be ignored

you assume that I'm sure what the answer is there

I'm guessing there's a way to require an element but ignore the value

I'm not even sure that it can have a known answer

there is, yes

I've got a bunch of time later today to dig into this in detail, so I can hopefully provide coherent feedback rather than piecemeal reactions

ok great

Back quickly to .expansion.identifier, this is what I'm seeing:

Group simple-cases
  Test simple-expand-all: Fail
    string property values differ at .expansion.identifier
Expected :$uuid$
Actual  :4aa6f81f-ab79-41b2-96e2-6faa0aadc38c

well, that's not a valid value

oh! it needs the urn:uuid: prefix?

yes

But the type is uri? which can be absolute or relative

well... a URI can be, but in this case:

uniquely identifies this expansion of the valueset

I think it should be absolute

There's several places in the spec where we missd this when we allowed relative URIs

uniquely in what scope though? wrt that specific tx server endpoint, or globally, or in some deployment environment?
I don't think you can legitimately enforce it to be a UUID (it might be something like [base]/expansion/[UUID], which would be "unique" and absolute)

This one is perhaps tricky:
Several tests expect an expansion parameter for excludeNested but Ontoserver always behaves as if this was true, and so omits it because its value does not affect Ontoserver's behaviour.

That’s less than ideal from my pov. and probably excludes ontoserver from serving for hl7 igs. Maybe. I’ll think about the testing ramifications. Is that fact visible in the terminology capabilities statement?

You have it as a uuid anyway, so prefixing isn’t going to be a problem? And the intent is global since expansions are sometimes cached and reused. Sometimes at scale

Globally unique is fine, but then I'd be tempted to adopt a URI based on the template [base]/expansion/[UUID] e.g., https://tx.ontoserver.csiro.au/expansion/4aa6f81f-ab79-41b2-96e2-6faa0aadc38c.
But in principle, if the spec says URI, unique identifier, then I don't think it's good form to impose additional constraints.

Ontoserver does return TerminologyCapabilities.expansion.hierarchical = false

But the meaning of excludeNested is only about the result representation (true => MUST return a flat expansion), it does not affect the logical content of the expansion.
Is there a reason you think that parameter should be included?

Conversely, Ontoserver redundantly includes offset and count values in the expansion parameters even if they haven't had any impact on paging.

I fully expect we'll need to do some adjustments in this space

Is there a reason you think that parameter should be included?

IG Authors have raised issues before when the expansion in the IG loses the heirarchy

@Michael Lawley I've been thinking about this one:

omits it because its value does not affect Ontoserver's behaviour.

That's wrong - the parameters are to inform a consumer how the value set was expanded. Whether or not Ontoserver can or can't is not the point, it's how it acted when doing the expansion

Ontoserver redundantly includes offset and count values in the expansion parameters even if they haven't had any impact on paging

offset = 0, presumably, but what's count in that case?

But the presence/absence/value of excludeNested doesn't affect "expansion" (i.e., which codes are present), it only potentially affects how those codes are returned in the ValueSet.expansion.contains.

Grahame Grieve said:

Ontoserver redundantly includes offset and count values in the expansion parameters even if they haven't had any impact on paging

offset = 0, presumably, but what's count in that case?

MAXINT

it still affects the expansion even if it doesn't affect which codes are present

If a consumer is looking through a set of expansions, instead of just generating a new one, then it's going to be input into their choice

I had been approaching it from the perspective of judging whether or not a persisted expansion is re-usable for a different expansion request.
(Which is something that Ontoserver does when it has a ValueSet with a stored expansion.)

indeed, but you're only thinking of it in your context, it could/would also be done in expansion users that can't make the assumption you're making

I'm trying to think about this from the perspective of a client / consumer of ValueSet.expansion -- under what circumstances do they need to know excludeNested = true? What is it actually telling them?

One answer might be "this value was provided for this expansion parameter in the original request"?

that this is expansion will not contain nested contains even if that might be relevent for this value set

Also, what should Ontoserver do if the request was $expand?excludeNested=false? Should it state that in the parameters even though the actual expansion may have (if it was present) flattened any nesting? Or, should it change it to true because flattening might have happened.

Perhaps the message is just "as a client, you do not have to look for nested codes when processing this expansion"?

well I think that the server should return an exception if the client asked it to do something it can't do

But that's not what excludeNested=false means. It's not the same as saying "include nested"

no that's true

and you don't know whether flattening is a thing that happened or not, I presume

correct

Now looking at all the validation test cases, the system parameter has the wrong type (valueString not valueUri) and, in the responses, code also has the wrong type (valueString instead of valueCode)

and similarly for system in the responses

wow that's a bad on my part. Fixed

nearly - still problems with the system parameter

diff --git a/tx/validation/simple-code-bad-code-request-parameters.json b/tx/validation/simple-code-bad-code-request-parameters.json
index 077c424..59d292a 100644
--- a/tx/validation/simple-code-bad-code-request-parameters.json
+++ b/tx/validation/simple-code-bad-code-request-parameters.json
@@ -8,6 +8,6 @@
     "valueCode" : "code1x"
   },{
     "name" : "system",
-    "valueString" : "http://hl7.org/fhir/test/CodeSystem/simple"
+    "valueUri" : "http://hl7.org/fhir/test/CodeSystem/simple"
   }]
 }

In validation/simple-code-implied-good-request-parameters.json, there is a non-standard parameter implySystem:

{
  "resourceType" : "Parameters",
  "parameter" : [{
    "name" : "url",
    "valueUri" : "http://hl7.org/fhir/test/ValueSet/simple-all"
  },{
    "name" : "code",
    "valueCode" : "code1"
  },{
    "name" : "implySystem",
    "valueBoolean" : true
  }]
}

indeed there is, and there should be, right?

it indicates that it is intentional that there's no system and the server should infer what the system is

But that is an invented non-standard parameter?
The use-case here seems to be that the system isn't knowable by the calling client, but in the context of validation, why wouldn't the system be known; there should be bindings available?

it's a code type, so there's only a code, and the server is asked to imply the system from the code and the value set

agree I haven't proposed that parameter, but it's still needed

Yes, its a code type, but that must exist in some context, right? The context should provide the system?

the value set itself is the context

What are the boundaries here? Can the ValueSet contain codes from > 1 code system? Can the code be non-unique in the valueset expansion?

The value set can contain codes from more than one code system, yes. A. number of them do. The code must be unique in the value set else it's an error

Presumably the system parameter does also need to be provided (from the documentation of $validate-code.code: "the code that is to be validated. If a code is provided, a system or a context must be provided"). Does the client just pass a dummy system that is ignored?

no the system is not provided in this case

since there isn't one

and yes, that violates the documentation on that parameter

And is it only ever used when supplying the code parameter?

yes. it must be accompanied by a code and a value set

I fixed the remaining system parameters

for examples like validation/simple-code-bad-display-response-parameters.json why is the result true when the display is invalid? The specification for the result output parameter is:

True if the concept details supplied are valid

Another test case issue: mis-named input parameter. See, for example, validation/simple-code-bad-version1-request-parameters.json which includes a parameter version that should instead be systemVersion.

why is the result true when the display is invalid?

Because that's just a warning - the code has been judged to be in the value set

a parameter version that should instead be systemVersion

ouch

why is the result true when the display is invalid?

Because that's just a warning - the code has been judged to be in the value set

But if a display is provided it should be validated and if its not any of the displays listed by the CodeSystem, then it is invalid -- the definition of result is not "True if the code is a member of the ValueSet/CodeSystem", but rather "if the concept details supplied are valid"; display is one of these details.

I am very uncomfortable about relaxing display validation affecting the outcome due to the prevalence of EHRs that allow for the display to be edited arbitrarily.

well, I'm very sure that if I changed to an error instead of a warning, the IG authoring community would completely rebel, but I guess TI might want to have an opinion. So what do other people think?

There are lots of reasons for display not being valid. (E.g. If someone has a code system supplement the validator doesn't know about.)

Why is the IG authoring community using non-valid displays?

there's 4 reasons that I've seen:

• it's close but not completely correct (it's been 'improved' a little)
• They're using the value set display, or an implied code system supplement
• they got the name from somewhere else and it's wrong there (VSAC). Often this is due to drift
• they think that the existing display is wrong and they wanted it to be right

1. Use the ValueSet or a supplement to "improve".
2. ValueSet display should succeed, as should explicit supplement; implicit/implied supplements should also work.
3. Then validation will tell them what an acceptable display is.
4. Use the ValueSet or a supplement to "fix".

Note, I am more concerned about the clinical community than the IG community.

If this is an impasse, perhaps the mode flag should be used to relax things?
Either way, I think we need an explicitly agreed mechanism to use the "issues" to flag the invalid display text.

Also, I think the test extensions-echo-all is wrong at least in assuming supplements will be automagically included

ValueSet display should succeed

TI decided otherwise; that's no longer allowed

I expect that TI will choose to decide this in NOLA. You going to be there?

Either way, I think we need an explicitly agreed mechanism to use the "issues" to flag the invalid display text.

the tests are doing that now

This has been discussed, at some length, with regard to SNOMED CT descriptions, and I recall that @Dion McMurtrie produced a table with various permutations in the early days of SNOMED on FHIR.

Unless the edition and version of SCT is provided, it's not possible to determine the validity of an unrecognized description. Otherwise, the best a server can do is return the preferred term from its default edition & version and a warning.

well, this discussion is not just about SCT that's for sure

Also, I think the test extensions-echo-all is wrong at least in assuming supplements will be automagically included

why?

That's precisely the intent of this test - make sure that supplements such as this are automagically included

language supplements

Grahame Grieve said:

well, this discussion is not just about SCT that's for sure

Sure - but things are a lot more straightforward for single edition, single language Code Systems.

that's not much of hill to climb given how complex SCT is

It's far more complex with things like LOINC where the same complexity (different national editions and local extensions) exists, but where everyone does it differently and often poorly.

Re extensions-echo-all, the supplement contains extensions (some I think are technically not valid where they're being used), and then expecting corresponding property values in the output (eg weight)

which ones are not valid?

ItemWeight - only goes on Coding and in a Questionnaire

I think I created a task about that one

I used a property where I could, and an extension where I had to

We can force the overhead of a CodeSystem supplement, but we can't count on the supplement being available when performing production-time validation. And that means that non-matching display names shouldn't be treated as an error.

If you're doing prod time validation without all the base info, then you're only going to get half answers - do you tolerate missing profiles? But, if your use case is tolerant of bad displays, just omit them from the validate-code calls, or let's have an explicit parameter that the client passes telling the server to only treat as warnings

@Michael Lawley to increase your happiness, I'm just adding tests for supporting these 3 parameters from $expand for $validate-code: system-version, check-system-version, force-system-version, and as I'm doing that, I'm checking that they apply to Coding.version as well

One of the other challenges with the txTests is that Ontoserver returns additional expansion.parameters and this causes the test to report a false failure

Most implementations don't care about the display values - and will be sloppy with them. So the default behavior should be warnings - errors should require the explicit parameter.

One of the other challenges with the txTests is that Ontoserver returns additional expansion.parameters and this causes the test to report a false failure

I'm assuming that this is something we'll sort out, so I'm not worrying about that today

but it's a test problem, not an implementation problem

Given that displays are what clinicians see and interpret, being sloppy is bad -- we've seen real clinical risks here.
And just because (a group of) ppl are sloppy doesn't mean we should enable that by default.

but it's a test problem, not an implementation problem

It's a test problem yes, but it's making it very hard for me to work through the cases because it bails out early and hides potential actual problems in the rest of the response.

fair.

Do you have a list of the extra parameters? In general, some extra parameters would be fine but others might not be, and I don't want to simply let anything go by

The reality is that the displays in many code systems are not appropriate for clinician display. By 'sloppy' I mean that systems make the displays what they need to be for appropriate user interface, not worrying too much about diverging from the 'official' display names if the 'official' names aren't useful for the purpose. I'm not saying that the display names chosen are typically inappropriate/wrong.

Do you have a list of the extra parameters?
version is the main one, and it seems strange that it's not expected in the result

Also, I'm getting a missing error for includeDesignations. Again, this seems like our interpretations of "parameters that affected expansion" are mis-aligned. I interpret this as being the calculation of the matching codes, not the specific representation that gets returned (noting that displayLanguage is counted since it affects the computed display value)

Coding.display A representation of the meaning of the code in the system, following the rules of the system.

"following the rules of the system", not "following the rules of some system implementer".

Also, if a display is not appropriate, then get it fixed -- either at source (in HL7 / THO) or with the external party. If the external party won't play ball, then fix it in a shared supplement so everyone can benefit rather than lots of (potentially incompatible) fixes spread over many different IGs.

it sounds so easy when you say it like that

Sure. Except that's not what systems do today. They just load the codes into their databases and make the display names say what they want them to say. And they're not going to change that just because we might like them to.

If that's all they did I'd be less concerned. What they REALLY DO is allow people to change the display text on-the-fly to absolutely anything (and people do this), and the results sometimes bear zero resemblance to the code's meaning. This is why I say we're concerned about the clinical use case over the IG use case, and why I want the caller to explicitly request that an invalid display not return an error; then the onus is on the caller.

version is the main one, and it seems strange that it's not expected in the result

where is it missing? I just spent a while hunting for it, and yes, it was missing from the validate-code results, but I can't see where it's missing from the $expand results

Let's start with simple/simple-expand-all-response-valueSet.json -- it only has:

    "parameter" : [{
      "name" : "excludeNested",
      "valueBoolean" : true
    }],

.. and ..?

Where is the version of the CodeSystem that was used in the expansion?

that code system doesn't have a version, so there's no parameter saying what it is

These days I guess that should be called system-version? But it's a canonical, so I would expect http://hl7.org/fhir/test/CodeSystem/simple| as the value

really? I would not expect that

That says "I use a version-less instance of this code system", rather than just not saying anything.

so firslty, it's not system-version - that's something else, an instruction about the default version to use. version is the actual version used. Though I just spent 15min verifying that for myself, and it could actually be documented

At least it's "not wrong"
+1 for documenting these :)

That says "I use a version-less instance of this code system", rather than just not saying anything

I

I'm not sure that it does. I just read the section on canonicals again, and at least we can say that this is not clear

FHIR-41165

I don't see another way to say it -- the trailing | might be optional, but is, I think, in the spirit of things?

I think that the IG publisher would blow up on this:

"parameter" : [{
      "name" : "version",
      "valueCanonical" : "http://hl7.org/fhir/test/CodeSystem/simple|"
    }]

If you want a versionless canonical, you omit the '|'. I would expect (and have only ever seen) the '|' there if there's a trailing version.

no wouldn't blow up, just wouldn't make sense in the page, because the code makes the same assumption as Lloyd

Hmm, that looks like it might be HAPI behaviour -- I'm guessing if you set the version to "" rather than null.
Investigating...

Yep, that is the issue.

Would IG publisher cope sensibly without the trailing |

    "parameter" : [{
      "name" : "version",
      "valueCanonical" : "http://hl7.org/fhir/test/CodeSystem/simple"
    }]

it'll ignore that one. As it will ignore http://hl7.org/fhir/test/CodeSystem/simple| from the next release

if there's no version, there's nothing to say

And I'll work around HAPI to leave off the |

but you won't leave the parameter out?

what about in the response to $validate-code when there's no version on the code system?

that's why you should leave it out

I'll have to look & think deeper - if the ValueSet has two code systems but one has no version, then it could be misleading / confusing to have only one "version" reported? I think leaving it out means clients may have to work harder.

why would clients have to work harder?

Just looking now at extensions/expand-echo-bad-supplement-parameters.json -- we've used PROCESSING as the code rather than BUSINESS-RULE ; seems a somewhat arbitrary distinction

It is but I don't mind changing

clients (that care) have to know that a missing version means a code-system didn't have a version. And, they have to scan the expansion to find all the code systems in scope (and this may not be a complete set if paging).
Additionally, what if the valueset references two "versions" of the same code system, and one is empty...hmm, not sure if that is possible with Ontoserver.

Re PROCESSING vs BUSINESS-RULE, ideally the test would allow either

what if the valueset references two "versions" of the same code system, and one is empty

You should go bang on that case

clients (that care) have to know that a missing version means a code-system didn't have a version

But they have to scan to decide that either way

Not if all the code systems are listed directly in expansion.parameters."version".
Another edge case - a code system is referenced in the compose, but no codes actually match - you'd never know it was in scope

Regarding setting the ValueSet.language to the value of $expand's displayLanguage parameter, will this not be misleading if only some of the codes have translations in the requested language?

sticking to version for now... you're really using it as more than a version - you're using it as a dependency list

I'm thinking that clients might be doing that, yes

well, if we're going to use it to report things that don't contain versions, then we should change it's name. Or would you not consider that?

Regarding setting the ValueSet.language to the value of $expand's displayLanguage parameter, will this not be misleading if only some of the codes have translations in the requested language?

possibly, it that's what was going on, but it's not

well, the tests now have version as optional

though I think we should consider renaming it

did you want to talk about other parameters before we talk about language?

and going back, I sure don't understand this:

Also, I'm getting a missing error for includeDesignations. Again, this seems like our interpretations of "parameters that affected expansion" are mis-aligned. I interpret this as being the calculation of the matching codes, not the specific representation that gets returned (noting that displayLanguage is counted since it affects the computed display value)

what's it got to do with the calculation of matching codes?

https://github.com/hapifhir/org.hl7.fhir.core/pull/1246 - work to date if you won't want to wait for some weird testing thing to be resolved

Ignore the includeDesignations thing - I'm just including it if a value was supplied.

(deleted)

Back on display validation, the example in the spec suggests that the appropriate response is to fail:
http://www.hl7.org/fhir/valueset-operation-validate-code.html#examples

Is there appetite for adding another mode, e.g. ALLOW_INVALID_DISPLAY ?

the example certainly does suggest failure is appropriate

As a status update, I think we're very close to passing except for the errors relating to unexpected "version" values which manifest like:

Group simple-cases
  Test simple-expand-all: Fail
    array properties count differs at .expansion.parameter
Expected :1
Actual  :2

and also some spurious validation of the actual error message strings:

  Test validation-simple-codeableconcept-bad-system: Fail
    string property values differ at .parameter[0].resource.issue[0].details.text

and

  Test validation-simple-codeableconcept-bad-version1: Fail
    string property values differ at .parameter[0].resource.issue[0].details.text

I figured the question of the actual error messages would come up at some point

but good to hear, thanks

Is there appetite for adding another mode, e.g. ALLOW_INVALID_DISPLAY ?

I don't think I'd like to add another mode for this. Or at least, not this alone. I'm considering the ramifications of just saying that's an error, and then picking through the issues in the IG publisher and downgrading it to a warning if the issues are only about displays.

Either way, I'll be putting this question to the two communities (TI and IG editors) in New Orleans

I think we're very close to passing

Well, too soon :-)
Seems the test harness complains about Ontoserver including extensions.
It also doesn't account for the expansion.contains being flat when excludeNested is not true.

But I believe these are txTests issues, not Ontoserver issues

A new spec issue -- expansion.parameter.value[x] doesn't support canonical only uri.
image.png

Which means the test responses that have an expansion.parameter like:

    {
      "name" : "version",
      "valueCanonical" : "http://hl7.org/fhir/test/CodeSystem/simple|0.1.0"
    }],

are invalid.

yeah I discovered that last night. I'm midway through revising them for other reasons and then I'll make another commit

@Michael Lawley I committed fixed up tests.

with regard to error messages, can you share a copy of the different error messages with me? I'm going to set the tests up so that the messages have to contain particular words. (I think)

I'm going to set the tests up so that the messages have to contain particular words. (I think)

Um, ok.

The specified code 'code1x' is not known to belong to the specified code system 'http://hl7.org/fhir/test/CodeSystem/simple'
A version for a code system with URL http://hl7.org/fhir/test/CodeSystem/simple was not supplied and the system could not find its latest version.
A version for a code system with URL http://hl7.org/fhir/test/CodeSystem/simplex was not supplied and the system could not find its latest version.
None of the codes in the codeable concept were valid.
The provided code "#code1x" was not found in value set http://hl7.org/fhir/test/ValueSet/simple-all
The provided code "http://hl7.org/fhir/test/CodeSystem/en-multi#code1" exists in the ValueSet, but the display "Anzeige 1" is incorrect
The provided code "http://hl7.org/fhir/test/CodeSystem/simple#code2a" was not found in value set http://hl7.org/fhir/test/ValueSet/simple-filter-regex

Another test case error:

• validation-simple-code-good-display

The ValueSet specifies a version for the code system 1.0.0 but the display value supplied in the request "good-display" is that from version 1.2.0 AND the response says that version 1.2.0 was used in the validation.

I think that's fixed up now?

No - https://github.com/FHIR/fhir-test-cases/blob/master/tx/validation/simple-code-good-display-response-parameters.json still shows version 1.2.0, last updated 20 hrs ago

but what's the request?

duh. I forgot to push :sad:

and now the request has valueString not valueUri for the system :man_facepalming:

ah, that's an ongoing issue -- I just have local changes to work around :-)

I'll fix

ok pushed

Thanks! At least with my test harness the main outstanding issues is the display validation issue.

Now looking at extensions-echo-enumerated:

1. Why are the top-level ValueSet.extension in the output expansion ValueSet?
2. (Not just for this expansion, but all) - ValueSet.compose, ValueSet.date, and ValueSet.publishershould all be optional.

the display validation issue?

whether an invalid display causes result to be false

oh right. yes

Why are the top-level ValueSet.extension in the output expansion ValueSet?

Because they might matter, so the server should echo them

Not just for this expansion, but all) - ValueSet.compose, ValueSet.date, and ValueSet.publishershould all be optional.

I guess. I don't think it matters to me? I'll check if I care

Why are the top-level ValueSet.extension in the output expansion ValueSet?
Because they might matter, so the server should echo them

That suggests a stronger link between specification and the expansion than I expect. This appears to be the key statement from 4.9.8 Value Set Expansion

A resource that represents a value set expansion includes the same identification details as the definition of the value set

What is the scope of "identification details"?

regarding ValueSet.compose: I have a parameter includeCompose for whether it should be returned or not, but I don't ever use it, and I wouldn't currently miss the compose

Is that not what includeDefinition is for?

Also, looking at the OperationOutcomes, why use .details.text rather than .diagnostics (given that there's no .details.coding values)

dear me it is

diagnostics is for things like stack dumps etc. The details of the issue go in details.text

That suggests a stronger link between specification and the expansion than I expect. This appears to be the key statement from 4.9.8 Value Set Expansion

I didn't understand that

What is the scope of "identification details"?
url + version + identifiers, I think

OperationOutcome.issue.diagnostics
Comment: This may be a description of how a value is erroneous [...]

But happy to update - it's all new

Stronger link...

Why would an extension on a ValueSet definition be relevant to its expansion (as a general rule)?

it shouldn't be but it might be relevant to the usage of the expansion

hence why I echo it

Hmm, ok

Should that be a requirement here?

no, in fact, they are only included if includeDefinition is true.

pushed new tests. code for running the tests is in the gg-202305-more-tx-work2 branch of core

my local copy of tx-fhir-org still fails one of the tests... might have more work to do on the tester

open issues - text details, + the display validation question which is going to committee in New Orleans

So, turns out that it is HAPI's code that's populating the OperationOutcome and putting the text into diagnostics and not details.text

This is only in the case of things like code system (supplement) or value set not found/resolvable since that's a 404 response

this one definitely matters.

Yep, I'll have to take over from the default interceptor behaviour

Thanks @Grahame Grieve I have the new tests and the gg-202305-more-tx-work2 branch running locally.
A bunch of tests are failing because the expected expansion is hierarchical, but Ontoserver returns a flat expansion so there are errors like:

Group parameters
  Test parameters-expand-all-hierarchy: Fail
    array properties count differs at .expansion.contains
Expected :3
Actual  :7

so why is Ontoserver returning a flat expansion? does it need a parameter?

Because it's allowed to, and unless you're returning "all codes", it's a hard problem to cut nodes out of a tree/graph

Let alone order them

but that one is all codes

All codes is very low on our priority list (infrequent use case) so we haven't done special-case work to preserve hierarchy.
It's also something that we've rarely been asked about.

it's certainly come up from the IG developers

and I'm surprised... structured expansions are a real thing for UI work

What we have heard is that some people want to have an explicit hierarchy on expansion that doesn't match the code system's hierarchy (eg where things are grouped differently from the normal isa hierarchy). In these cases the simplest approach we've found is to have them express the desired hierarchy in the stored expansion.

that might be, but as you see, there's reasons people want a heirarchy

But for IG developers, why do they care about the (on the wire) expansion; if the IG tooling needs to render the hierarchy, then it's in the CodeSystem already, or can be recovered from the ValueSet with $expand?property=parent.

it IG tooling defers to the tx service on this matter. It doesn't try to impose heirachy on what the tx server chooses to return

@Michael Lawley we're going to do triage here on our open issues tomorrow. What I have in my mind:

• validating display - warning or error if display is wrong
• whether supplements are automatically in scope or not
• How a server reports that it doesn't do heirarchical expansions
• should we rename the 'version' parameter in .expansion.parameters

have I missed anything?

Wrt "How a server reports that it doesn't do heirarchical expansions", a server may do this in some circumstances but not others. For example, Ontoserver (currently) does not do them when calculating the expansion itself, but may return them if its (re-)using a stored expansion.

Other examples like this would be where, for example, a ValueSet includes the implicit all-codes ValueSet and the result is flat, but if it instead includes the CodeSystem directly then it is hierarchical

it IG tooling defers to the tx service on this matter. It doesn't try to impose heirachy on what the tx server chooses to return

Then it's effectively choosing to be happy with what the tx server returns, and in that case anything that is in-spec with the general FHIR tx services spec should be acceptable.

Then it's effectively choosing to be happy with what the tx server returns, and in that case anything that is in-spec with the general FHIR tx services spec should be acceptable.

It is acceptable from the infrastructure's pov, but not acceptable from the consumer's pov

it might be acceptable to some consumers, the ones who choose to use Ontoserver, but I think that would mean many editors would not be ok with HL7 using Ontoservrer

Other examples like this would be where, for example, a ValueSet includes the implicit all-codes ValueSet and the result is flat, but if it instead includes the CodeSystem directly then it is hierarchical

but that's how the test case we're talking about works

Other examples like this would be where, for example, a ValueSet includes the implicit all-codes ValueSet and the result is flat, but if it instead includes the CodeSystem directly then it is hierarchical

If $expand?url=vs1 returns a hierarchical expansion, then I define vs2 as "include vs1", should it not also return a hierarchical expansion?

It is acceptable from the infrastructure's pov, but not acceptable from the consumer's pov

From my perspective, the consumer here is using a tool that could provide this behaviour itself by using the CodeSystem directly (or by reconstructing the hierarchy from parent relationships), but the tool chooses to hand it off to the tx server. Since this is a context-specific behaviour, why not have the tool that wants it, implement it?

Of course, if Ontoserver users call for this behaviour, then that's something we would strong consider, but otherwise it seems like there's an undocumented set of use-cases where a specific behaviour is desired that we have to discover in a trial by error manner.

well, here you are, discovering it :grinning:

returning an hierarchical expansion when the value set includes all of a hierarchical code system is a required feature for HL7 IG publication

Probably because I'm grounded in HL7 culture, but for me that's totally obvious and hardly needs to be stated as a requirement, so there you go. However, Ontoserver doesn't need to do that to be used by the eco-system as an additional terminology server

I'm thinking about how to handle that in the tests - that's why I asked whether this is a feature that surfaces in the metadata anywhere. But it doesn't :sad:

other than parameters-expand-all-hierarchy, parameters-expand-enum-hierarchy, and parameters-expand-isa-hierarchy, does this affect any other tests?

on the subject of display error/warning, I'll be advocating for a parameter that defaults to leaving the tx server returning an error.

is it another mode flag? or something else?

I think another mode flag works. With the default being return error, and the flag saying don't error on displays, just warn.

FHIR-41229

I've just updated https://r4.ontoserver.csiro.au/fhir with the work-in-progress changes to align better with the requirements as expressed in the txTests
I believe that many of the reported failures are false negatives, and some are very hard to understand what's going on, e.g.:

  Test validation-simple-code-good-version:   ... Exception: Error from server: Error:org.hl7.fhir.r4.model.CodeableConcept@11b455e5

    org.hl7.fhir.r4.utils.client.EFhirClientException: Error from server: Error:org.hl7.fhir.r4.model.CodeableConcept@11b455e5

    at org.hl7.fhir.r4.utils.client.network.FhirRequestBuilder.unmarshalReference(FhirRequestBuilder.java:263)
    at org.hl7.fhir.r4.utils.client.network.FhirRequestBuilder.execute(FhirRequestBuilder.java:230)
    at org.hl7.fhir.r4.utils.client.network.Client.executeFhirRequest(Client.java:194)
    at org.hl7.fhir.r4.utils.client.network.Client.issuePostRequest(Client.java:119)
    at org.hl7.fhir.r4.utils.client.FHIRToolingClient.operateType(FHIRToolingClient.java:279)
    at org.hl7.fhir.convertors.txClient.TerminologyClientR4.validateVS(TerminologyClientR4.java:137)
    at org.hl7.fhir.validation.special.TxTester.validate(TxTester.java:252)
    at org.hl7.fhir.validation.special.TxTester.runTest(TxTester.java:191)
    at org.hl7.fhir.validation.special.TxTester.runSuite(TxTester.java:163)
    at org.hl7.fhir.validation.special.TxTester.execute(TxTester.java:95)
    at org.hl7.fhir.validation.ValidatorCli.parseTestParamsAndExecute(ValidatorCli.java:227)
    at org.hl7.fhir.validation.ValidatorCli.main(ValidatorCli.java:148)

I'll investigate

it's sure not a useful error message

FHIR-41230

I noticed also that the test fixtures are not automatically created?
Also language/codesystem-de-multi.json has elements like title:en which fails when I tried to load it in (using the 5->4 converter in HAPI)

oh. right

you can't use those directly, no

I forgot - I was playing around with that format and left it in

in the case of that test, the error should be

 Error from server: Error:[0a8c6743-42a8-43fe-bca5-1138aa91595d]: Could not find value set http://hl7.org/fhir/test/ValueSet/version-all-1 and version null. If this is an implicit value set please make sure the url is correct. Implicit values sets for different code systems are specified in https://www.hl7.org/fhir/terminologies-systems.html.

I noticed also that the test fixtures are not automatically created?

I'm not sure what that means

All the test code systems, and valuesets identified in test-cases.json are not automatically loaded into Ontoserver when I run the txTests thing. Instead, I needed to run my own loader

no they're passed in a tx-resource parameter with each request

I didn't notice this until just now, running against the new r4.ontoserver deployment since previously I was testing against a local server that I'd already loaded things onto

Aha! Another magic parameter -- is support for that part of the test?

this is already known. You and I discussed it in the past. see FHIR-33944. It's very definitely required

The test cases do it this way since support is required to support the IG publisher

https://github.com/hapifhir/org.hl7.fhir.core/pull/1255 for the execution problem

Yes, I recall the proposal.

The test cases do it this way since support is required to support the IG publisher

that's effectively what I was asking.

Does this also extend to FHIR-33946 and the cache-id parameter?

that one is optional - the client looks in the capability statement to see if cache-id is stated to be supported before deciding that the server is capable of doing that

though the test cases don't try that

I'm going to have to put some considered thought into how we support tx-resource.
Non-exhaustive list of considerations:

1. write permissions (or lack thereof),
2. operating in read-only mode (common for national / reference servers),
3. name clashes
4. visibility and isolation

None of these are a problem for us with ValueSet resources (we already support contained ValueSets), but they are for CodeSystems.

for me, those are not a thing - they are never written. You probably can't avoid that. But what's 'name clashes' about?

What happens when the resource passed via tx-resource has the same URL as one that is already on the server? Does it shadow it? It may have an older version than the one on the server and the reference from the request may not be version-specific; should the older version supplied via tx-resource be preferred over the newer one?

here's what I drafted about that:

One or more additional resources that are referred to from the value set provided with the $expand or $validate-code invocation. These may be additional value sets or code systems that the client believes will or may be necessary to perform the operation. Resources provided in this fashion are used preferentially to those known to the system, though servers may return an error if these resources are already known to the server (by URL and version) but differ from that information on the server

@Michael Lawley I updated the test cases for the new mode parameter

Thanks. I note that it is still complaining about extension content (Ontoserver includes some of its own extensions). I would have expected addition extension content to be generally ignored?

which extensions?

Michael Lawley said:

Coding.display A representation of the meaning of the code in the system, following the rules of the system.

"following the rules of the system", not "following the rules of some system implementer".

Also, if a display is not appropriate, then get it fixed -- either at source (in HL7 / THO) or with the external party. If the external party won't play ball, then fix it in a shared supplement so everyone can benefit rather than lots of (potentially incompatible) fixes spread over many different IGs.

Remembering that things get done according to the path of the least resistance, I see very little instruction and zero examples of using supplements in http://hl7.org/fhir/valueset.html - so chances of them being used for this purpose are very slim. Any changes in this area must offer a path of less or at most equal resistance compared to trimming the display text to what you mean.

well, we can provide examples, that's for sure.

Yep, at the same time, there is dragon text on the supplements:

The impact of Code System supplements on value set expansion - and therefore value set validation - is subject to ongoing experimentation and implementation testing, and further clarification and additional rules might be proposed in future versions of this specification.

That would need to go away as well to get confidence in using them

Otherwise hard to say 'this is what you shall use' when it's an experimental thing.

we're coming out of the experimentation phase :grinning:

and talking about the additional rules

Michael Lawley said:

If that's all they did I'd be less concerned. What they REALLY DO is allow people to change the display text on-the-fly to absolutely anything (and people do this), and the results sometimes bear zero resemblance to the code's meaning. This is why I say we're concerned about the clinical use case over the IG use case, and why I want the caller to explicitly request that an invalid display not return an error; then the onus is on the caller.

I don't see how this will improve the situation. It would just become an almost mandatory thing you do "just because the spec requires it" and it wouldn't carry the intended meaning.

Good use of supplements would, that way the IG can be explicit about the display codes it is tweaking to better fit the purpose. I'd be happy to do that in my IGs!

@Michael Lawley I finally got to a previously reported issue:

However, I'm trying to use tx.fhir.org/r4 as a reference point but I can't get it to behave.
For example http://tx.fhir.org/r4/ValueSet/$validate-code?system=http://snomed.info/sct&code=22298006&url=http://snomed.info/sct?fhir_vs=isa/118672003 gives a result=true even though the code is not in the valueset. In fact the url parameter seems to be totally ignored?

Indeed. It's an issue in the parser because there's 2 = in the parameter - it's splitting on the second not the first

it works as expected if you escape the second =

I believe the correct strategy is to take the query part (everything from the 1st ?) and split on &, then split each of these on the first = only

I didn't say I was happy with what it's doing

ah, not your parser code then?

it is. it's the oldest code I have. I think I haven't touched it since 1997 or so

PR time?

maybe. The URL itself is invalid so the behaviour isn't wrong, but I don't like it much

https://github.com/HealthIntersections/fhirserver/commit/eee300e80d1a0f4ed56ceba99972253e2dc0df97

Why is that URL invalid?

an unescaped = in it. I think that's not valid according to the http spec. But I upgraded the server anyway, and it should be OK now

according to https://www.rfc-editor.org/info/rfc3986 it is valid, and '=' is considered to be a subdilimiter.

that doesn't really relate to it's use in key/value pairs

I don't see where an unescaped = is illegal?

@Michael Lawley a new issue has raised it's ugly head.

consider the situation where a value set refers to an unknown code system, and just includes all of it, and a client asks to validate the code

e.g.

{
  "resourceType" : "ValueSet",
  "id" : "unknown-system",
  "url" : "http://hl7.org/fhir/test/ValueSet/unknown-system",
  "version" : "5.0.0",
  "name" : "UnknownSystem",
  "title" : "Unknown System",
  "status" : "active",
  "experimental" : true,
  "date" : "2023-04-01",
  "publisher" : "FHIR Project",
  "compose" : {
    "include" : [{
      "system" : "http://hl7.org/fhir/test/CodeSystem/simpleX"
    }]
  }
}

and

{
  "resourceType" : "Parameters",
  "parameter" : [{
    "name" : "url",
    "valueUri" : "http://hl7.org/fhir/test/ValueSet/unknown-system"
  },{
    "name" : "code",
    "valueCode" : "code1"
  },{
    "name" : "system",
    "valueUri" : "http://hl7.org/fhir/test/CodeSystem/simpleX"
  }]
}

This is a pretty common situation in the IG world, and the IG publisher considers this a warning not an error.

but it's very clearly an error validating

... only... the validator decides that this is one of those cases because there's a parameter

  "cause" : "not-found"

where cause is taken from OperationOutcome.issue.type.

but I removed cause from the returned parameters, and now I have no way to know that the valueset validation failed because of an unknown code system

the case above says that there is an unknown code system, but it doesn't explicitly say that the result is false because of the unknown code system.

This is a "fail to validate" rather than a "validate = false" situation -- I'd expect a 4XX series error from the Tx and an OperationOutcome about the CodeSystem not found.
Will that work?
I'm pretty sure Ontoserver does something like this

I don't think that's right - other issues can still be detected and returned

So I don't follow why you have removed cause?

it wasn't a standard parameter. And it was pretty loose anyway

it's kind of weird to just put 'cause : not found' and assume everyone knows that means validation failed because the code system needed to determine value set membership wasn't found

I need a better way to say it...

you also have location: ["code.system"] and the details.text

I do have that, but I'm going to be second guessing the server to decide whether that's the cause, or an incidental finding

Does this come down to identifying which one (or more?) of the issues was the trigger for result = false?

yes that's one way to look at it

Can it be as simple as "all the issues with severity = error"?

no I don't think it can. There's plenty of scope of issues with severity = error whether or not the code is in the value set

Doesn't that depend on how you interpret things? For example, if validating a codeableConcept, then you validate each contained Coding. If they all fail, then each contributes an issue with severity of error, but if any passes, then the issues from the others would just be warning?

This seems to be in line with

Indicates how relevant the issue is to the overall success of the action

I certainly don't think levels work like that. If a system is wrong, or a code is invalid, then that's an error

at the local level, but not at the level of the overall operation

issue.code has this comment:

For example, code-invalid might be a warning or error, depending on the context

really?

really

Comments:
Code values should align with the severity. For example, a code of forbidden generally wouldn't make sense with a severity of information or warning. Similarly, a code of informational would generally not make sense with a severity of fatal or error. However, there are no strict rules about what severities must be used with which codes. For example, code-invalid might be a warning or error, depending on the context

(my emphasis)

oh I believed you. And I probably did write that. But I've noodled on it for a couple of hours, and in the context of the validator, invalid codes are invalid codes, whether they're in the scope of the value set or not.

and on further noodling, I think this is OK to be an extension for tx.fhir.org - the notion of 'it's not an error because the code system is unknown' is kind of centric to the base tx service, and not to additional ones. So I'm going with a parameter name of x-caused-by-unknown-system for the link, and the tests won't require that

@Michael Lawley https://github.com/FHIR/fhir-test-cases/blob/master/tx/requirements.md

also @Jonathan Payne

@Grahame Grieve Looks nice... :+1:

other/codesystem-dual-filter.json is invalid -- it has a duplicate code: AA

Also, HAPI is complaining about language/codesystem-de-multi.json:

HAPI-0450: Failed to parse request body as JSON resource. Error was: HAPI-1825: Unknown element 'title:en' found during parse

hmm

hapi probably doesn't support JSON 5 either. can you try commenting that line out?

So, the testing/comparison aspect is complaining about / rejecting extensions that Ontoserver includes that are not part of the expected result.
e.g.,

Group simple-cases
  Test simple-expand-all: Fail
    properties differ at .expansion.contains[1]: missing property extension
  Test simple-expand-enum: Fail
    properties differ at .expansion.contains[1]: missing property extension
  Test simple-expand-isa: Fail
    properties differ at .expansion.contains[0]: missing property extension
  Test simple-expand-prop: Fail
    properties differ at .expansion.contains[0]: missing property extension
  Test simple-expand-regex: Fail
    properties differ at .expansion.contains[1]: missing property extension

what extensions are yo including?

One is http://ontoserver.csiro.au/profiles/expansion

what is it?

Why does that matter? It's an extension, if you don't understand it you can (should) ignore it.

(It's actually legacy from DSTU2_1 to indicate inactive status)

it doesn't matter for the tests, no, but I'm just interested for the sake of being nosy

:laughing:

I'll think about the testing part

@Michael Lawley https://github.com/hapifhir/org.hl7.fhir.core/pull/1303

I have rewritten these two pages:

• https://confluence.hl7.org/display/FHIR/Publishing+terminology+to+the+FHIR+Ecosystem
• https://github.com/FHIR/fhir-test-cases/blob/master/tx/requirements.md

I have removed the section on registration - I'm rewriting that after talking to @Michael Lawley, more on that soon

I reconciled the two pages and changed the way the web source reference works

@Grahame Grieve Hi, I am running the fhir tx testsuite against Snowstorm. For some tests, there are complaints about a missing "id" property, and the test fails. Turns out that the resource that is returned contains an "id" whereas the "reference" resource does not contain an "id". Is this a real "fail", or is the "id" property supposed to be optional?
Expected:

{
  "$optional-properties$" : ["date", "publisher", "compose"],
  "resourceType" : "ValueSet",
  "url" : "http://hl7.org/fhir/test/ValueSet/simple-all",
  "version" : "5.0.0",
  "name" : "SimpleValueSetAll",
  "title" : "Simple ValueSet All",
  "status" : "active",
  "experimental" : false,
  "date" : "2023-04-01",
  "publisher" : "FHIR Project",
  "compose" : {
    "include" : [{
      "system" : "http://hl7.org/fhir/test/CodeSystem/simple"
    }]
  },
  "expansion" : {
    "identifier" : "$uuid$",
    "timestamp" : "$instant$",
    "total" : 7,
    "parameter" : [{
      "name" : "excludeNested",
      "valueBoolean" : true
    },
    {
      "name" : "used-codesystem",
      "valueUri" : "http://hl7.org/fhir/test/CodeSystem/simple|0.1.0"
    },
    {
      "$optional$" : true,
      "name" : "version",
      "valueUri" : "http://hl7.org/fhir/test/CodeSystem/simple|0.1.0"
    }],
    "contains" : [{
      "system" : "http://hl7.org/fhir/test/CodeSystem/simple",
      "code" : "code1",
      "display" : "Display 1"
    },
    {
      "system" : "http://hl7.org/fhir/test/CodeSystem/simple",
      "abstract" : true,
      "inactive" : true,
      "code" : "code2",
      "display" : "Display 2"
    },
    {
      "system" : "http://hl7.org/fhir/test/CodeSystem/simple",
      "code" : "code2a",
      "display" : "Display 2a"
    },
    {
      "system" : "http://hl7.org/fhir/test/CodeSystem/simple",
      "code" : "code2aI",
      "display" : "Display 2aI"
    },
    {
      "system" : "http://hl7.org/fhir/test/CodeSystem/simple",
      "code" : "code2aII",
      "display" : "Display 2aII"
    },
    {
      "system" : "http://hl7.org/fhir/test/CodeSystem/simple",
      "code" : "code2b",
      "display" : "Display 2b"
    },
    {
      "system" : "http://hl7.org/fhir/test/CodeSystem/simple",
      "code" : "code3",
      "display" : "Display 3"
    }]
  }
}

Actual:

{
    "resourceType": "ValueSet",
    "id": "simple-all",
    "url": "http://hl7.org/fhir/test/ValueSet/simple-all",
    "version": "5.0.0",
    "name": "SimpleValueSetAll",
    "title": "Simple ValueSet All",
    "status": "active",
    "experimental": false,
    "publisher": "FHIR Project",
    "expansion": {
        "id": "f4b71bf6-3ef4-4c30-a4ea-ab3f4ae3dad6",
        "timestamp": "2024-10-09T15:08:23+02:00",
        "total": 7,
        "offset": 0,
        "parameter": [
            {
                "name": "version",
                "valueUri": "http://hl7.org/fhir/test/CodeSystem/simple|0.1.0"
            },
            {
                "name": "displayLanguage",
                "valueString": "en"
            }
        ],
        "contains": [
            {
                "system": "http://hl7.org/fhir/test/CodeSystem/simple",
                "code": "code1",
                "display": "Display 1"
            },
            {
                "system": "http://hl7.org/fhir/test/CodeSystem/simple",
                "code": "code2",
                "display": "Display 2"
            },
            {
                "system": "http://hl7.org/fhir/test/CodeSystem/simple",
                "code": "code2a",
                "display": "Display 2a"
            },
            {
                "system": "http://hl7.org/fhir/test/CodeSystem/simple",
                "code": "code2aI",
                "display": "Display 2aI"
            },
            {
                "system": "http://hl7.org/fhir/test/CodeSystem/simple",
                "code": "code2aII",
                "display": "Display 2aII"
            },
            {
                "system": "http://hl7.org/fhir/test/CodeSystem/simple",
                "code": "code2b",
                "display": "Display 2b"
            },
            {
                "system": "http://hl7.org/fhir/test/CodeSystem/simple",
                "code": "code3",
                "display": "Display 3"
            }
        ]
    }
}

it's not an error to return a populated id element. If doesn't even have to the same id. Probably it shouldn't be, but that's a style question

which means that the test is wrong, really

I updated the tests to allow id, but you'll have to wait for the release of a new validator to use them, unfortunately

about 24 hours

As you may know from other messages, I am investigating the options to make Snowstorm fhir tx testsuite compliant. As our reference server for terminology in Belgium is an Ontoserver (now 6.20.1 since yesterday), and I want the Snowstorm behaviour to be as similar as possible to the Ontoserver behaviour, I also ran the fhir tx testsuite against Ontoserver. I got a result of 16% fails.

I know from #Announcements > Using Ontoserver with Validator / IG Publisher that Ontoserver is considered compatible. How should I interpret the 16% failed tests? Is any software allowed to fail 16% tests? Any 16%, or only that specific 16% of the tests? What is also strange, is that the highest amount of failures is in the "simple-cases" test group. Is the "simple-cases" test group the test of _basic_ behaviour, and are these tests of a greater weight? What does this say about the interplay between IGPublisher and the tested terminology server?

I don't know about 16% failure. What version are you running? I test the public ontoserver everyday and get 100% pass rate

https://tx.ontoserver.csiro.au/fhir

is that the highest amount of failures is in the "simple-cases" test group

hmm. maybe you need to set a parameter for flat rather than nested? Ontoserver doesn't do nested expansions, and that's a setting you pass to the test cases

try -mode flat

new Publication: STU 1 of theFHIR Shorthand implementation guide: http://hl7.org/fhir/uv/shorthand/STU1

New Publication: STU 1 of the FHIR Da Vinci Unsolicited Notifications Implementation Guide: http://hl7.org/fhir/us/davinci-alerts/STU1

New Publication: STU 1.1 of the C-CDA on FHIR Implementation Guide: http://hl7.org/fhir/us/ccda/STU1.1

New Publication: STU 1 of the Vital Records Mortality and Morbidity Reporting FHIR Implementation Guide: http://hl7.org/fhir/us/vrdr/STU1/index.html

New Publication: STU1 of the CARIN Consumer Directed Payer Data Exchange (CARIN IG for Blue Button®) FHIR Implementation Guide: http://hl7.org/fhir/us/carin-bb/STU1

New Publication: STU1 of the HL7 Payer Data Exchange (PDex) Payer Network, Release 1 - US Realm Implementation Guide: hl7.org/fhir/us/davinci-pdex-plan-net/STU1

New Publication: STU1 of the HL7 Prior-Authorization Support (PAS), Release 1- US Realm FHIR® Implementation Guide: http://hl7.org/fhir/us/davinci-pas/STU1

New Publication: STU1 of the HL7 Payer Data Exchange (PDex), Release 1 - US Realm FHIR® Implementation Guide: http://hl7.org/fhir/us/davinci-pdex/STU1

New Publication: STU1 of the HL7 Da Vinci - Coverage Requirements Discovery (CRD), Release 1- US Realm FHIR® Implementation Guide: http://hl7.org/fhir/us/davinci-crd/STU1

New Publication: STU1 of the HL7 FHIR® Implementation Guide: Payer Coverage Decision Exchange, R1 - US Realm: http://hl7.org/fhir/us/davinci-pcde/STU1

New Publication: STU1 of the FHIR® Implementation Guide: Documentation Templates and Payer Rules (DTR), Release 1- US Realm: http://hl7.org/fhir/us/davinci-dtr/STU1/index.html

New Publication: STU1 of the HL7 FHIR® Implementation Guide: Risk Based Contract Member Identification, Release 1 - US Realm: http://hl7.org/fhir/us/davinci-atr/STU1

New Publication: STU1 of the HL7 FHIR® Implementation Guide: Pharmacist Care Plan Document, Release 1 - US Realm: http://hl7.org/fhir/us/phcp/STU1

New Publication: STU1 of the HL7 FHIR® Implementation Guide: Clinical Guidelines, Release 1: http://hl7.org/fhir/uv/cpg/STU1

Newly Posted: FHIR R4B Ballot #1: http://hl7.org/fhir/2021Mar

New Publication: Normative Release 1 of the HL7 Cross-Paradigm Specification: Clinical Quality Language (CQL), Release 1: http://cql.hl7.org/N1

New Publication: STU Release 1 of the HL7/NCPDP FHIR® Implementation Guide: Specialty Medication Enrollment, Release 1 - US Realm: http://hl7.org/fhir/us/specialty-rx/STU1.

New Publication: STU Release 3 of HL7 FHIR® Implementation Guide: Data Exchange for Quality Measures STU3 for FHIR R4: http://hl7.org/fhir/us/davinci-deqm/STU3

Lynn Laakso said:

New Publication: STU Release 3 of HL7 FHIR® Implementation Guide: Data Exchange for Quality Measures STU3 for FHIR R4: http://hl7.org/fhir/us/cqfmeasures/STU3

http://hl7.org/fhir/us/davinci-deqm/STU3/

new Publication: STU Release 1 of the HL7 Immunization Decision Support Forecast (ImmDS) Implementation Guide: http://hl7.org/fhir/us/immds/STU1

New Publication: STU Release 4 of the HL7 FHIR® US Core Implementation Guide STU 4 Release 4.0.0: http://hl7.org/fhir/us/core/STU4

File not found ;-)

well that's not supposed to happen

it'll work now

The change log appears to be empty? http://hl7.org/fhir/us/core/history.html

Grahame has to fix that, it'll be 12 hours

fixed

New Publication: STU Update Release 1.1 of HL7 FHIR® Implementation Guide: Consumer Directed Payer Data Exchange (CARIN IG for Blue Button®), Release 1 - US Realm: http://www.hl7.org/fhir/us/carin-bb/STU1.1

I don't know as it matters but the directory of published versions doesn't show this version. http://hl7.org/fhir/us/carin-bb/history.html

it does for me. You might have a caching problelm

New Publication: STU Update Release 1.1 of HL7 FHIR® Profile: Occupational Data for Health (ODH), Release 1 - US Realm: http://hl7.org/fhir/us/odh/STU1.1

New Publication: STU Release 1 of HL7 FHIR® Implementation Guide: Vital Records Common FHIR Profile Library, Release 1: http://hl7.org/fhir/us/vr-common-library/STU1

New publication: STU Release 1 of HL7 FHIR® Implementation Guide: NHSN Inpatient Medication COVID-19 Administration Reports, Release 1- US Realm: http://hl7.org/fhir/us/nhsn-med-admin/STU1

New Publication: STU Release 1 of HL7 FHIR® Implementation Guide: NHSN Adverse Drug Event - Hypoglycemia Report, Release 1- US Realm: http://hl7.org/fhir/us/nhsn-ade/STU1

New Publication: STU Update (STU1.1) of HL7 FHIR® Implementation Guide: DaVinci Payer Data Exchange US Drug Formulary, Release 1 - US Realm: http://hl7.org/fhir/us/Davinci-drug-formulary/STU1.1

New Publication: STU Publication of HL7 FHIR® Implementation Guide: Vital Records Birth and Fetal Death Reporting, Release 1 - US Realm: http://hl7.org/fhir/us/bfdr/STU1

New Publication: STU Publication of HL7 FHIR® Implementation Guide: Dental Data Exchange, Release 1 - US Realm: http://hl7.org/fhir/us/dental-data-exchange/STU1

New Publication: STU Publication of HL7 FHIR® Implementation Guide: Post-Acute Care Cognitive Status, Release 1- US Realm: http://hl7.org/fhir/us/pacio-cs/STU1

New Publication: STU Publication of HL7 FHIR® Implementation Guide: Post-Acute Care Functional Status, Release 1- US Realm: http://hl7.org/fhir/us/pacio-fs/STU1

New Publication: Release 4.0.1 of the CQF FHIR® Implementation Guide: Clinical Quality Framework Common FHIR Assets: http://fhir.org/guides/cqf/common/4.0.1/. (note: this is not a guide published through the HL7 consensus process, but according to the FHIR Community Process, so it's posted on fhir.org)

STU Update Publication of HL7 FHIR® Implementation Guide: Prior-Authorization Support (PAS), Release 1- US Realm: http://hl7.org/fhir/us/davinci-pas/STU1.1

STU Publication of HL7 FHIR Implementation Guide: minimal Common Oncology Data Elements (mCODE) Release 1 STU 2 – US Realm: http://hl7.org/fhir/us/mcode/STU2

STU Publication of HL7 FHIR® Implementation Guide: Electronic Case Reporting (eCR), Release 2: http://hl7.org/fhir/us/ecr/STU2

STU Update Publication of HL7 FHIR® Profile: Quality, Release 1 STU 4.1- US Realm: http://hl7.org/fhir/us/qicore/STU4.1

STU Publication of HL7 FHIR Implementation Guide: Profiles for ICSR Transfusion and Vaccination Adverse Event Detection and Reporting, Release 1 - US Realm: www.hl7.org/fhir/us/icsr-ae-reporting/STU1

Normative Publication of HL7 FHIR® Implementation Guide: FHIR Shorthand, Release 2: http://hl7.org/fhir/uv/shorthand/N1

STU Publication of HL7 FHIR® Structured Data Capture (SDC) Implementation Guide, Release 3: http://hl7.org/fhir/uv/sdc/STU3

STU Publication of HL7 FHIR® Implementation Guide: Clinical Data Exchange (CDex), Release 1- US Realm: http://hl7.org/fhir/us/davinci-cdex/STU1

STU Publication of HL7 FHIR® Implementation Guide: Health Record Exchange (HRex) Framework, Release 1- US Realm: http://hl7.org/fhir/us/davinci-hrex/STU1

STU Errata Publication of HL7 FHIR® Profile: Quality, Release 1 - US Realm STU 4.1.1: http://hl7.org/fhir/us/qicore/STU4.1.1

@David Pyke and @John Moehrke are pleased to announce the release of HotBeverage #FHIR Implementation Guide release April 1st - Based on IETF RFC 2324 allows for the fulfillment of a device request for an artfully brewed caffeinated beverage. http://fhir.org/guides/acme/HotBeverage/1.4.2022

STU Update Publication for HL7 FHIR® Implementation Guide: Payer Data Exchange (PDex) Payer Network, Release 1 - US Realm: http://hl7.org/fhir/us/davinci-pdex-plan-net/STU1.1

STU Publication of HL7 FHIR® Implementation Guide: Quality Measures, Release 1 STU 3 - US Realm: http://hl7.org/fhir/us/cqf-measures/STU3

Informative Publication of HL7 EHRS-FM Release 2.1 – Pediatric Care Health IT Functional Profile Release 1 – US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=593

STU Publication of HL7 FHIR® IG: SMART Web Messaging Implementation Guide, Release 1: http://hl7.org/fhir/uv/smart-web-messaging/STU1

STU Publication of HL7 FHIR® Implementation Guide: Clinical Genomics, STU 2: http://hl7.org/fhir/uv/genomics-reporting/STU2

STU Publication of HL7 Domain Analysis Model: Vital Records, Release 5- US Realm: see http://www.hl7.org/implement/standards/product_brief.cfm?product_id=466

STU Publication of HL7 FHIR® Implementation Guide: Personal Health Device (PHD), Release 1: http://hl7.org/fhir/uv/phd/STU1

STU Publication of HL7 CDA® R2 IG: C-CDA Templates for Clinical Notes STU Companion Guide, Release 3 – US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=447

STU Publication of HL7 FHIR® US Core Implementation Guide STU5 Release 5.0.0: http://hl7.org/fhir/us/core/STU5

STU Update Publication of HL7 CDA® R2 Implementation Guide: National Health Care Surveys (NHCS), Release 1, STU Release 2.1 and STU Release 3.1 – US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=385

STU Publication of HL7 FHIR® Implementation Guide: Risk Adjustment, Release 1 - US Realm: http://hl7.org/fhir/us/davinci-ra/STU1

Informative Guidance Publication of HL7 Short Term Solution - V2: SOGI Data Exchange Profile: http://www.hl7.org/permalink/?SOGIGuidance

Errata Publication of CDA® R2.1 (HL7 Clinical Document Architecture, Release 2.1): https://www.hl7.org/documentcenter/private/standards/cda/2019CDAR2_1_2022JUNerrata.zip

Errata Publication of US Core STU5 Release 5.0.1: http://hl7.org/fhir/us/core/STU5.0.1

STU Publication of HL7 FHIR® Implementation Guide: Digital Insurance Card, Release 1 - US Realm: http://hl7.org/fhir/us/insurance-card/STU1

STU Publication of HL7 FHIR® Implementation Guide: Subscription R5 Backport, Release 1: http://hl7.org/fhir/uv/subscriptions-backport/STU1

STU Update Publication of HL7 CDA® R2 Implementation Guide: Reportability Response, Release 1 STU Release 1.1- US Realm: http://www.hl7.org/implement/standards/product_brief.cfm?product_id=470

STU Update Publication Request of HL7 CDA® R2 Implementation Guide: Public Health Case Report - the Electronic Initial Case Report (eICR) Release 2, STU Release 3.1 - US Realm: http://www.hl7.org/implement/standards/product_brief.cfm?product_id=436

Informative Publication of HL7 FHIR® Implementation Guide: COVID-19 FHIR Clinical Profile Library, Release 1 - US Realm: http://hl7.org/fhir/us/covid19library/informative1

STU Publication of HL7 CDA® R2 Implementation Guide: NHSN Healthcare Associated Infection (HAI) Reports, Release 4, STU 2 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=570

STU Publication of HL7 FHIR® Implementation Guide: Clinical Data Exchange (CDex), Release 1 STU1.1.0 - US Realm: http://hl7.org/fhir/us/davinci-cdex/STU1.1

STU Update Publication of HL7 FHIR Profile: Occupational Data for Health (ODH), Release 1.2: http://hl7.org/fhir/us/odh/STU1.2

STU Publication of HL7 FHIR® Implementation Guide: Payer Data Exchange (PDex) Drug Formulary, Release 1 STU2 - US Realm: http://hl7.org/fhir/us/davinci-drug-formulary/STU2

STU Publication of HL7 FHIR® Implementation Guide: Vital Records Death Reporting (VRDR), Release 1 STU2 - US Realm: http://hl7.org/fhir/us/vrdr/STU2

STU Update Publication of HL7 FHIR® Implementation Guide: Electronic Case Reporting (eCR), Release 2.1 - US Realm: http://hl7.org/fhir/us/ecr/STU2.1

R5 Ballot is published. http://hl7.org/fhir/2022Sep/

STU Publication of HL7 FHIR® Implementation Guide: Vital Signs, Release 1- US Realm: http://hl7.org/fhir/us/vitals/STU1/

STU Publication of HL7 Cross Paradigm Specification: CDS Hooks, Release 1: https://cds-hooks.hl7.org/2.0/

New release of HL7 Terminology (THO) v4.0.0: https://terminology.hl7.org/4.0.0. (Thanks @Marc Duteau)

STU Publication of HL7 FHIR® Implementation Guide: Hybrid/Intermediary Exchange, Release 1- US Realm: http://www.hl7.org/fhir/us/exchange-routing/STU1

Errata publication of C-CDA (HL7 CDA® R2 Implementation Guide: Consolidated CDA Templates for Clinical Notes - US Realm): https://www.hl7.org/implement/standards/product_brief.cfm?product_id=492

STU Publication of HL7 FHIR® Implementation Guide: Security for Registration, Authentication, and Authorization, Release 1- US Realm: http://hl7.org/fhir/us/udap-security/STU1/

STU Publication of HL7 FHIR® Implementation Guide: FHIR for FAIR, Release 1: http://hl7.org/fhir/uv/fhir-for-fair/STU1

STU Publication of HL7 FHIR® Implementation Guide: PACIO Re-assessment Timepoints, Release 1 - US Realm: http://hl7.org/fhir/us/pacio-rt/STU1

STU Publication of HL7 FHIR® Implementation Guide: Medicolegal Death Investigation (MDI), Release 1 - US Realm: http://hl7.org/fhir/us/mdi/STU1

STU Publication of HL7 CDA® R2 Implementation Guide: ePOLST: Portable Medical Orders About Resuscitation and Initial Treatment, Release 1 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=600

STU Publication of HL7 Version 2.5.1 Implementation Guide: Laboratory Results Interface, Release 1 STU Release 4 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=279

STU Publication of HL7 Version 2.5.1 Implementation Guide: Laboratory Orders (LOI) from EHR, Release 1, STU Release 4 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=152

STU Publication of HL7 Version 2 Implementation Guide: Laboratory Value Set Companion Guide, Release 2- US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=413

New release of HL7 Terminology (THO) v5.0.0: https://terminology.hl7.org/5.0.0

This also means that the THO freeze has been lifted.

You can view the UTG tickets that were implemented in this release using the following dashboard and selecting 5.0.0 in the first pie chart. https://jira.hl7.org/secure/Dashboard.jspa?selectPageId=16115

Informative Publication of HL7 V2 Implementation Guide Quality Criteria, Release 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=608

STU Publication of HL7 FHIR® Implementation Guide: SDOH Clinical Care, Release 2.0 - US Realm: http://hl7.org/fhir/us/sdoh-clinicalcare/STU2

STU Update Publication of HL7 FHIR® Implementation Guide: Data Exchange for Quality Measures, STU3.1 for FHIR R4 - US Realm: http://hl7.org/fhir/us/davinci-deqm/STU3.1/

STU Update Publication of HL7 FHIR® Implementation Guide: International Patient Summary, Release 1.1: http://hl7.org/fhir/uv/ips/STU1.1

STU Publication of HL7 FHIR® Implementation Guide: Consumer-Directed Payer Exchange (CARIN IG for Blue Button®), Release 1 STU2: http://hl7.org/fhir/us/carin-bb/STU2

STU Publication Request for HL7 Domain Analysis Model: Nutrition Care, Release 3 STU 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=609

Errata Publication of HL7 CDA® R2 Implementation Guide: Quality Reporting Document Architecture - Category I (QRDA I) - US Realm, STU 5.3: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=35

Snapshot3 of FHIR Core spec: http://hl7.org/fhir/5.0.0-snapshot3. This is published to support the Jan 2023 connectathon, and help prepare for the final publication of R5, which is still scheduled for March 2023

Informative Publication of HL7 EHRS-FM R2.0.1: Usability Functional Profile, Release 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=611

STU Publication of NHSN Healthcare Associated Infection (HAI) Reports Long Term Care Facilities (HAI-LTCF-FHIR), Release 1 - US Realm: http://hl7.org/fhir/us/hai-ltcf/STU1

STU Update Publication of HL7 FHIR® Implementation Guide: Subscription R5 Backport, Release 1, STU 1.1: http://hl7.org/fhir/uv/subscriptions-backport/STU1.1/

New release of HL7 Terminology (THO) v5.1.0: https://terminology.hl7.org/5.1.0

The Final Draft version of FHIR R5 is now published for QA : http://hl7.org/fhir/5.0.0-draft-final. There's a two week period to do QA on it. In particular, we'd like to focus on the invariants - there'll be another announcement about that shortly

STU Update Publication of minimal Common Oncology Data Elements (mCODE) Implementation Guide 2.1.0 - STU 2.1: http://hl7.org/fhir/us/mcode/STU2.1/

STU Update Publication of HL7 FHIR Profile: Occupational Data for Health (ODH), Release 1.3: https://hl7.org/fhir/us/odh/STU1.3/

STU Publication of HL7 FHIR® Implementation Guide: Clinical Data Exchange (CDex), Release 1 STU 2 - US Realm: http://hl7.org/fhir/us/davinci-cdex/STU2/

STU Update Publication of HL7 FHIR® Implementation Guide: Vital Records Death Reporting (VRDR), Release 1 STU2.1 - US Realm: https://hl7.org/fhir/us/vrdr/STU2.1/

I have started publishing R5. Unlike the IGs, R5 is rather a big upload - it will take me a couple of days. In the meantime, you might find discontinuities and broken links on the site, and confusion between R4 and R5 as bits are copied up. Also you may find missing and broken redirects too. I will make another announcement once it's all uploaded

STU Publication of HL7 FHIR® Implementation Guide: International Patient Access (IPA), Release 1: http://hl7.org/fhir/uv/ipa/STU1

STU Publication of HL7 FHIR® Implementation Guide: Longitudinal Maternal & Infant Health Information for Research, Release 1 - US Realm: http://hl7.org/fhir/us/mihr/STU1/

STU Publication of HL7 FHIR® Implementation Guide: Patient Cost Transparency, Release 1 - US Realm: http://hl7.org/fhir/us/davinci-pct/STU1

STU Publication of HL7 FHIR® Profile: Quality, Release 1 - US Realm (qicore) STU Release 5: http://hl7.org/fhir/us/qicore/STU5

Normative Publication of HL7 CDA® R2 Implementation Guide: Emergency Medical Services; Patient Care Report Release 3 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=438

STU Publication of HL7 Consumer Mobile Health Application Functional Framework (cMHAFF), Release 1, STU 2: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=476

STU Publication of HL7 FHIR® Implementation Guide: Data Segmentation for Privacy (DS4P), Release 1: http://hl7.org/fhir/uv/security-label-ds4p/STU1

STU Publication of HL7 FHIR® IG: SMART Application Launch Framework, Release 2.1: http://hl7.org/fhir/smart-app-launch/STU2.1

STU Publication of HL7 Version 2 Implementation Guide: Diagnostic Audiology Reporting, Release 1- US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=620

STU Publication of HL7 FHIR® R4 Implementation Guide: Clinical Study Schedule of Activities, Edition 1: http://hl7.org/fhir/uv/vulcan-schedule/STU1/

STU Update Publication of HL7 FHIR® Implementation Guide: NHSN Healthcare Associated Infection (HAI) Reports for Long Term Care Facilities (HAI-LTCF-FHIR), Release 1 STU 1.1 - US Realm: http://hl7.org/fhir/us/hai-ltcf/STU1.1

STU Publication of HL7 CDA® R2 Implementation Guide: Personal Advance Care Plan (PACP) Document, Edition 1, STU3 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=434

STU Publication of HL7 CDA® R2 Implementation Guide: Pharmacy Templates, Edition 1 STU Release 2: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=514

STU Publication of HL7 FHIR® R4 Implementation Guide: Single Institutional Review Board Project (sIRB), Edition 1- US Realm: http://hl7.org/fhir/us/sirb/STU1

STU Publication of HL7 CDA® R2 Implementation Guide: C-CDA Templates for Clinical Notes STU Companion Guide Release 4 - US Realm Standard for Trial Use May 2023: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=447

STU Publication of HL7 FHIR® US Core Implementation Guide STU6 Release 6.0.0: http://hl7.org/fhir/us/core/STU6

STU Publication of HL7/NCPDP FHIR® Implementation Guide: Specialty Medication Enrollment, Release 1 STU 2 - US Realm: http://hl7.org/fhir/us/specialty-rx/STU2/

STU Publication of Vulcan's HL7 FHIR® Implementation Guide: Retrieval of Real World Data for Clinical Research STU 1 - UV Realm: http://hl7.org/fhir/uv/vulcan-rwd/STU1

Version 6.1.0-snapshot1 of US Core for public review of forth coming STU update to STU6 - US Realm: http://hl7.org/fhir/us/core/STU6.1-snapshot1

STU Publication of HL7 FHIR® Implementation Guide: Military Service History and Status, Release 1 - US Realm: http://hl7.org/fhir/us/military-service/STU1

STU Publication of HL7 FHIR® Implementation Guide: Identity Matching, Release 1 - US Realm: http://hl7.org/fhir/us/identity-matching/STU1

STU Publication of HL7 FHIR® Implementation Guide: Making Electronic Data More Available for Research and Public Health (MedMorph) Reference Architecture, Release 1- US Realm: http://hl7.org/fhir/us/medmorph/STU1/

STU Update Publication of HL7 CDA® R2 Implementation Guide: National Healthcare Safety Network (NHSN) Healthcare Associated Infection (HAI) Reports for Long Term Care Facilities (HAI-LTCF-CDA), Release 1, STU 1.1 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=546

STU Update Publication of HL7 CDA® R2 Implementation Guide: C-CDA Templates for Clinical Notes Companion Guide, Release 4.1 STU - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=447

STU Update Publication of HL7 FHIR® US Core Implementation Guide STU6 Release 6.1.0: http://hl7.org/fhir/us/core/STU6.1

STU Publication of HL7 FHIR® Implementation Guide: Cancer Electronic Pathology Reporting, Release 1 - US Realm: https://hl7.org/fhir/us/cancer-reporting/STU1

STU Publication of HL7 FHIR Implementation Guide: Electronic Medicinal Product Information, Release 1: http://hl7.org/fhir/uv/emedicinal-product-info/STU1

Unballoted STU Update Publication of HL7 FHIR® Implementation Guide: SDOH Clinical Care, Release 2.1 - US Realm: http://hl7.org/fhir/us/sdoh-clinicalcare/STU2.1

STU Publication of HL7 FHIR® Implementation Guide: CodeX™ Radiation Therapy, Release 1- US Realm: http://hl7.org/fhir/us/codex-radiation-therapy/STU1

STU Publication of HL7 FHIR® Implementation Guide: US Public Health Profiles Library, Release 1 - US Realm: http://hl7.org/fhir/us/ph-library/STU1

STU Publication of HL7 FHIR® Implementation Guide: ICHOM Patient Centered Outcomes Measure Set for Breast Cancer, Edition 1: http://hl7.org/fhir/uv/ichom-breast-cancer/STU1

STU Publication of HL7 FHIR® Implementation Guide: Health Care Surveys Content, Release 1 - US Realm: http://hl7.org/fhir/us/health-care-surveys-reporting/STU1

STU Publication of HL7 FHIR® Implementation Guide: Physical Activity, Release 1 - US Realm: http://hl7.org/fhir/us/physical-activity/STU1

STU Publication of HL7 FHIR® Implementation Guide: Quality Measures, Release 1 STU4 - US Realm: http://hl7.org/fhir/us/cqfmeasures/STU4

Unballoted STU Update Publication of HL7 FHIR® Implementation Guide: Healthcare Associated Infection Reports, Release 1, STU 2.1 —US Realm: http://hl7.org/fhir/us/hai/STU2.1

STU Publication of HL7 Cross Paradigm Specification: Health Services Reference Architecture (HL7-HSRA), Edition 1:https://www.hl7.org/implement/standards/product_brief.cfm?product_id=632

Errata publication of HL7 CDA® R2 Attachment Implementation Guide: Exchange of C-CDA Based Documents, Release 2 US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=464

Informative Publication of HL7 EHR-S FM R2.1 Functional Profile: Problem-Oriented Health Record (POHR) for Problem List Management (PLM), Edition 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=630

STU Publication of HL7 CDA R2 Implementation Guide: Gender Harmony - Sex and Gender representation, Edition 1 - Component of: HL7 Cross-Paradigm Implementation Guide: Gender Harmony - Sex and Gender representation, Edition 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=633

Informative Publication of HL7 Cross-paradigm Implementation Guide: Gender Harmony - Sex and Gender Representation, Edition 1: http://hl7.org/xprod/ig/uv/gender-harmony/informative1

STU Publication of HL7 FHIR® Implementation Guide: Data Exchange for Quality Measures, Edition 1 STU4 - US Realm: http://hl7.org/fhir/us/davinci-deqm/STU4

STU Publication of HL7 FHIR® Implementation Guide: Human Services Directory, Release 1 - US Realm: http://hl7.org/fhir/us/hsds/STU1

STU Update Publication of HL7 CDA® R2 Implementation Guide: NHSN Healthcare Associated Infection (HAI) Reports, Release 4, STU 2.1 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=570

STU Update Publication of HL7 FHIR® Implementation Guide: Vital Records Common FHIR Profile Library R1.1: http://hl7.org/fhir/us/vr-common-library/STU1.1

Errata:

I wrong wrote:

STU Publication of HL7 Cross-Product Implementation Guide: HL7 Cross Paradigm Implementation Guide: Gender Harmony - Sex and Gender Representation, Edition 1: http://hl7.org/xprod/ig/uv/gender-harmony/

This was a copy paste error on my part, sorry. This is an informative publication, not a trial-use publication

STU Update Publication of HL7 FHIR® Implementation Guide: Vital Records Birth and Fetal Death Reporting, Release 1.1: http://hl7.org/fhir/us/bfdr/STU1.1

STU Update Publication of Vital Records Death Reporting FHIR Implementation Guide, STU2.2 - US Realm: http://hl7.org/fhir/us/vrdr/STU2.2

STU Publication of HL7 FHIR® Implementation Guide: Coverage Requirements Discovery, Edition 2 - US Realm: http://hl7.org/fhir/us/davinci-crd/STU2

STU Publication of HL7 FHIR Implementation Guide: minimal Common Oncology Data Elements (mCODE) Release 1 STU 3 - US Realm: http://hl7.org/fhir/us/mcode/STU3

STU Publication of HL7 FHIR® Implementation Guide: Documentation Templates and Rules, Edition 2 - US Realm: http://hl7.org/fhir/us/davinci-dtr/STU2

STU Update Publication of HL7 CDA R2 Implementation Guide: Personal Advance Care Plan (PACP), Edition 1 STU 3.1 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=434

STU Publication of HL7 FHIR® Implementation Guide: Protocols for Clinical Registry Extraction and Data Submission (CREDS), Release 1 - US Realm: http://hl7.org/fhir/us/registry-protocols/STU1

Informative Publication of HL7 Informative Document: Patient Contributed Data, Edition 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=638

STU Update Publication of HL7 FHIR® Implementation Guide: Medicolegal Death Investigation (MDI), Release 1.1 - US Realm: http://hl7.org/fhir/us/mdi/STU1.1

STU Publication of HL7 FHIR® Implementation Guide: Prior-Authorization Support (PAS), Edition 2 - US Realm: http://hl7.org/fhir/us/davinci-pas/STU2

FHIR Foundation Publication: HRSA 2023 Uniform Data System (UDS) Patient Level Submission (PLS) (UDS+) FHIR IG, Release 1- see http://fhir.org/guides/hrsa/uds-plus/

HL7 DK Publication: DK Core version 3.0 is now published at https://hl7.dk/fhir/core/index.html

STU Publication of Health Level Seven Arden Syntax for Medical Logic Systems, Edition 3.0: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=639

STU Publication of HL7 FHIR® Implementation Guide: Integrating the Healthcare Enterprise (IHE) Structured Data Capture/electronic Cancer Protocols on FHIR, Release 1- US Realm: http://hl7.org/fhir/uv/ihe-sdc-ecc/STU1

1st Draft Ballot of HL7 FHIR® R6: http://hl7.org/fhir/6.0.0-ballot1

Release of HL7 FHIR® Tooling IG (International): http://hl7.org/fhir/tools/0.1.0

Ballot for the next versions of the FHIR Extensions Pack (5.1.0-ballot1): http://hl7.org/fhir/extensions/5.1.0-ballot/

Ballot for CCDA 3.0.0: http://hl7.org/cda/us/ccda/2024Jan/

This is a particularly important milestone for the publishing process. Quoting from the specification itself:

Within HL7, since 2020, an initiative to develop the same underlying publication process tech stack across all HL7 standards has been underway. The intent is to provide the same look and feel, to leverage inherent validation and versioning, to ease annual updates, and to avoid the unwieldy word and pdf publication process. This publication of C-CDA R3.0 is the realization of that intent for the CDA product family.

Many people have contributed to this over a number of years, and while I'm hesitant to call attention to any particular individuals because of the certainty of missing some others who also deserve it, it would not have got across the line without a significant contribution from @Benjamin Flessner

Informative Publication of HL7 FHIR® Implementation Guide: Record Lifecycle Events (RLE), Edition 1: http://hl7.org/fhir/uv/ehrs-rle/Informative1

STU Update Publication of HL7 FHIR® Implementation Guide: Patient Cost Transparency, Release 1 - US Realm: http://hl7.org/fhir/us/davinci-pct/STU1.1

STU Publication of HL7 FHIR® Implementation Guide: PACIO Personal Functioning and Engagement, Release 1 - US Realm: http://hl7.org/fhir/us/pacio-pfe/STU1

STU Publication of HL7 FHIR® Implementation Guide: Payer Data Exchange (PDex), Release 2 - US Realm: http://hl7.org/fhir/us/davinci-pdex/STU2

STU Publication of HL7 FHIR® Implementation Guide: Member Attribution List, Edition 2- US Realm: http://hl7.org/fhir/us/davinci-atr/STU2

STU Publication of HL7 FHIR® Implementation Guide: PACIO Advance Directive Interoperability, Edition 1 - US Realm: http://hl7.org/fhir/us/pacio-adi/STU1

STU Publication of HL7 FHIR® R4 Implementation Guide: QI-Core, Edition 1.6 - US Realm: http://hl7.org/fhir/us/qicore/STU6

STU Update Publication of HL7 CDA® R2 Implementation Guide: NHSN Healthcare Associated Infection (HAI) Reports, Release 4, STU 2.2 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=570

Interim Snapshot 5.1.0-snapshot1 of the Extensions package (hl7.fhir.yv.extensions#5.1.0-snapshot1) has been published to support publication requests waiting for a new release of the extensions package @ http://hl7.org/fhir/extensions/5.1.0-snapshot1/

STU Publication of HL7 FHIR® Implementation Guide: C-CDA on FHIR, STU 1.2.0 - US Realm: http://hl7.org/fhir/us/ccda/STU1.2

STU Update Publication of HL7 CDA® R2 Implementation Guide: National Healthcare Safety Network (NHSN) Healthcare Associated Infection (HAI) Reports for Long Term Care Facilities (HAI-LTCF-CDA), Release 1, STU 1.2 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=546

STU Publication of HL7 CDS Hooks: Hook Library, Edition 1: https://cds-hooks.hl7.org/

STU Publication ofHL7 FHIR® R5 Implementation Guide: Adverse Event Clinical Research, Edition 1: http://hl7.org/fhir/uv/ae-research-ig/STU1

STU Update Publication of HL7 FHIR® Implementation Guide: Digital Insurance Card, Release 1 - US Realm: http://hl7.org/fhir/us/insurance-card/STU1.1/

STU Publication of HL7 FHIR® R4 Implementation Guide: Adverse Event Clinical Research R4 Backport, Edition 1: http://hl7.org/fhir/uv/ae-research-backport-ig/STU1

STU Update Publication of HL7 FHIR® Implementation Guide: Central Cancer Registry Reporting Content IG, Edition 1- US Realm: https://hl7.org/fhir/us/cancer-reporting/STU1.0.1

STU Publication of HL7 FHIR® Implementation Guide: SMART Application Launch Framework, Release 2.2: http://hl7.org/fhir/smart-app-launch/STU2.2

STU Publication of HL7 FHIR® Implementation Guide: Pharmaceutical Quality (Industry), Edition 1: http://hl7.org/fhir/uv/pharm-quality/STU1

STU Publication of HL7 FHIR® US Core Implementation Guide STU 7 Release 7.0.0 - US Realm: http://hl7.org/fhir/us/core/STU7

STU Publication of HL7 Version 2.5.1 Implementation Guide: Laboratory Orders from EHR (LOI) Edition 5 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=152

STU Publication of HL7 Version 2.5.1 Implementation Guide: Laboratory Results Interface (LRI), Edition 5 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=279

Ok, a significant milestone has been reached with two new publications:

• CDA Core on StructureDefinitions Release 2.0.0-sd: http://hl7.org/cda/stds/core/
• CCDA R3 (3.0.0): http://hl7.org/cda/us/ccda/

STU Publication of the HL7 FHIR® R4 Implementation Guide: Electronic Long-Term Services and Supports (eLTSS) Edition 1 STU2 - US Realm: http://hl7.org/fhir/us/eltss/STU2

STU Publication of HL7 CDA® R2 Implementation Guide: NHSN Healthcare Associated Infection (HAI) Reports for Antimicrobial Use in Long Term Care Facilities (AULTC), Edition 1.0, STU1 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=646

STU Publication of HL7 FHIR® Implementation Guide: Central Cancer Registry Reporting Content IG, Edition 1- US Realm: http://hl7.org/fhir/us/central-cancer-registry-reporting/STU1

STU Publication of HL7 FHIR® Implementation Guide: Using CQL With FHIR, Edition 1: http://hl7.org/fhir/uv/cql/STU1

STU Publication of HL7 FHIR® Implementation Guide: Canonical Resource Management Infrastructure (CRMI), Edition 1: http://hl7.org/fhir/uv/crmi/STU1

STU Publication of HL7 FHIR® Implementation Guide: Value Based Performance Reporting (VBPR), Edition 1 - US Realm: http://hl7.org/fhir/us/davinci-vbpr/STU1

STU Update Publication of HL7 FHIR® R4 Implementation Guide: At-Home In-Vitro Test Report, Edition 1.1: http://hl7.org/fhir/us/home-lab-report/STU1.1

STU Publication of MCC eCare Plan Implementation Guide, Edition 1 - US Realm: http://hl7.org/fhir/us/mcc/STU1

Normative Reaffirmation Publication of HL7 Version 3 Standard: Event Publish & Subscribe Service Interface, Release 1 - US Realm and HL7 Version 3 Standard: Unified Communication Service Interface, Release 1 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=390 or https://www.hl7.org/implement/standards/product_brief.cfm?product_id=388

Normative Reaffirmation Publication of HL7 Version 3 Standard: Regulated Studies - Annotated ECG, Release 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=70

Normative Reaffirmation Publication of Health Level Seven Arden Syntax for Medical Logic Systems, Version 2.10: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=372

Normative Reaffirmation Publication of HL7 Healthcare Privacy and Security Classification System, Release 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=345

Normative Reaffirmation Publication of HL7 EHR Clinical Research Functional Profile, Release 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=16

Normative Reaffirmation Publication of HL7 EHR Child Health Functional Profile, Release 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=15

Normative Reaffirmation Publication of HL7 Version 3 Standard: XML Implementation Technology Specification - Wire Format Compatible Release 1 Data Types, Release 1 and HL7 Version 3 Standard: XML Implementation Technology Specification - V3 Structures for Wire Format Compatible Release 1 Data Types, Release 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=357 and https://www.hl7.org/implement/standards/product_brief.cfm?product_id=358

Normative Reaffirmation Publication of HL7 Version 3 Standard: Privacy, Access and Security Services; Security Labeling Service, Release 1: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=360

Reaffirmation Publication of HL7 Version 3 Implementation Guide: Context-Aware Knowledge Retrieval Application (Infobutton), Release 4: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=22

Normative Publication of HL7 FHIR® Implementation Guide: FHIR Shorthand, Edition 3.0.0: http://hl7.org/fhir/uv/shorthand/N2

STU Publication Request for HL7 FHIR® Implementation Guide: Medication Risk Evaluation and Mitigation Strategies (REMS), Edition 1- US Realm: http://hl7.org/fhir/us/medication-rems/STU1

Normative Reaffirmation Publication of HL7 Cross-Paradigm Specification: FHIRPath, Release 1: http://hl7.org/FHIRPath/N2

STU Update Publication of HL7 FHIR® Implementation Guide: Security for Registration, Authentication, and Authorization (FAST), Edition 1 - US Realm: http://hl7.org/fhir/us/udap-security/STU1.1

Informative Publication of HL7 Guidance: AI/ML Data Lifecycle, Edition 1 - US Realm: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=658

Unballoted STU Update of HL7 FHIR® Implementation Guide: SDOH Clinical Care, Release 2.2 - US Realm: http://hl7.org/fhir/us/sdoh-clinicalcare/STU2.2

Normative Publication of HL7 Clinical Document Architecture R2.0 Specification Online Navigation, Edition 2024: https://hl7.org/cda/stds/online-navigation/index.html

Normative Publication of Health Level Standard Standard Version 2.9.1 - An Application Protocol for Electronic Data Exchange in Healthcare Environments: https://www.hl7.org/implement/standards/product_brief.cfm?product_id=649

STU Publication of HL7 FHIR® Implementation Guide: Vital Records Common Library, Edition 2 - US Realm: http://hl7.org/fhir/us/vr-common-library/STU2

Normative Retirement Publication of HL7 V3 Patient Registry R1, Person Registry R1, Personnel Management R1 and Scheduling R2: Patient Registry, Person Registry, Personnel Management and Scheduling.

@Ward Weistra My colleague tried grab us.core#7.0.0 package from https://packages.fhir.org/hl7.fhir.us.core The package lists up to 6.1.0 but does not have 7.0.0. I tried using direct url: https://packages.simplifier.net/hl7.fhir.us.core/7.0.0 which also shows that this version does not exist. How should I get us core v7 package into simplifier?
Thanks and have a safe travel back home.

I had the same issue, looking at the package on https://build.fhir.org/ig/HL7/US-Core/downloads.html, in the package.json file we have:

{
  "name" : "hl7.fhir.us.core",
  "version" : "7.0.0",
  "tools-version" : 3,
  "type" : "IG",
  "date" : "20240627054756",
  "license" : "CC0-1.0",
  "canonical" : "http://hl7.org/fhir/us/core",
  "notForPublication" : true,

with the bottom line ensuring it isn't published on the package registry. I guess the only option is to download it manually

@Eric Haas is this a publication bug?

I don't have control over package creation. The link is the same as in the publication history page. Simplifier only lists 6.1.0 and 3.1.1. Where are you downloading it manually?

Here: https://hl7.org/fhir/us/core/STU7/package.tgz

@Ryan May @Eric Haas I think that may just be a result of looking at build.fhir.org. The package registry gets released versions from https://github.com/FHIR/ig-registry/blob/master/package-feeds.json -> https://hl7.org/fhir/package-feed.xml and the one there looks fine.

@Yunwei Wang The real issue is here: US Core 7 builds on VSAC 0.18.0 and that package is huge, so has been refused by the package registry infrastructure for now. US Core will next be refused because of missing dependencies.

The consensus is now that VSAC should indeed no longer be distributed as a FHIR package, but we're investigating if an exception can be made for the existing VSAC packages, perhaps one or more future ones and only those. And agree at FHIR-I on a package size limit value.

But this will take a moment...

See #tooling > package registry

(deleted)

The versioned US Core all point to the correct package version, The current version points to the current package, so I think this will be all sorted out in the next versions. :fingers_crossed:

IN case it helps - for the short term, any VSAC value set that is used in US Core (or C-CDA) has HL7 US Realm Program Management Author and HL7 US Realm Program Management Steward (Role : Steward). Should be part of the VS meta data

(deleted)

Also - minimally - IMHO the package should include only value sets that have status "active"

OK for the bigger issue though - I think my points stand. Also we could probably find out all IGs that use VSAC for VS build and source of truth and find out who the authors/stewards are and limit the VSAC package to that

@Grahame Grieve I'll continue to explore whether we can still load and serve VSAC 0.18.0+ for now.

But Gay has a suggestion above for a logical filtering of VSAC. Would this work for upcoming releases at least until a VSAC FHIR server is set up? For new VSAC packages it would be clear for users if they are missing a VS/CS when validating their IG.

(Or I'd welcome doing that retroactively for VSAC 0.18.0 and up too. Potentially we could run checks for all packages you know to depend on VSAC 0.18.0+ if they miss anything)

I'm going to investigate

here's my data:

we use value sets from the following stewards:

• ADVault
• AHA
• AHRQ CDS Connect
• AND
• BSeR
• CDC NCHS
• CSTE Steward
• CareEvolution Steward
• HL
• HL7 USRPM
• IMPAQ
• Lantana
• MITRE
• NCQA
• OPA
• Optum
• SAMHSA Steward
• The Joint Commission
• eCR

full-data.txt

so @Gay Dolin's suggestion does not work

image.png

indeed, but does that make any difference?

I guess these could still be 19 separate packages. If need be, the next iteration of the VSAC package could depend on all of those.

I have no idea how many VS/CS those Stewards have in VSAC in total, but if that's a manageable amount you could include all for a Steward in such a subpackage so you don't need a new edition when someone needs one more.

hl7.fhir.us.vsac.hl7-usrpm for example.

or simplifer could simply allow bigger packages, which would be way easier for everyone

How big are the 19 together.? I think that could be "The" package.

There is no need to pull in all the other sets - so many are really poorl value sets
US Realm Program mgt (US Core/C-CDA sets could REALLY take advantage of a seperate package. We possibly could then do away with depending on VSAC for the "Annual Releases" .
It would save HL7 about $40,000 a year, and ONC possibly even more

if people have used those 19, why would they not be allowed to use others?

Generally its IG authors who are building the sets

There is no need to pull in all the other sets - so many are really poorl value sets

if you can get formal agreement from #terminology that no one has a valid reason to use any other other stewards, I can remove them, sure

US Realm Program mgt (US Core/C-CDA sets could REALLY take advantage of a seperate package. We possibly could then do away with depending on VSAC for the "Annual Releases" . It would save HL7 about $40,000 a year, and ONC possibly even more

I don't know anything about annual releases, but US realm could just use THO or define it's own package. I'm told that VSAC is used because it's a better authoring environment

It is a better authoring environment

Heading into SD but will love to chat more about why this WOULD work

Grahame Grieve said:

if you can get formal agreement from #terminology that no one has a valid reason to use any other other stewards, I can remove them, sure

Terminology would never agree to that, as they should not. But perhaps the short term solution is using those current stewards, so publishing can get going again

Maybe long term solution is making simplifier bigger, but maybe each package update could still be based on: 1) IGs that have VSAC value sets 2) who are the stewards and then it will always be managable

If not the above, If we only pulled in value sets with status "Active" and some already published IG have sets that have flipped to "Not Maintained" - will that be problematic wrt to validation in implmentations?

If not - we could just pull in "active" sets - but that would still be pretty large - since VSAC now forces maintenance (https://www.nlm.nih.gov/vsac/support/authorguidelines/valuesetstatus.html) (though I'm guessing most of the "crap" value sets stewards/authors don't pay attenton to these emails that warn you your set is getting flagged as not maintained)

WRT the C-CDA annual release (which includes the shared US Core sets whose source of truth is VSAC), Since 2016 HL7/ONC has provided an annual release, basically to make up for the fact that all of C-CDA had not been balloted since 2015: https://vsac.nlm.nih.gov/download/ccda

At a cost to HL7 and ONC

We were hoping to get VSAC to create an ability for small releases to "push a button" and create a release, but they said "maybe we could start working on that in 2026"

So, @Brett Marquard and I are exploring if vendors (or vendor customers) even need a "release" and/or if we could offer it another way, hl7.fhir.us.vsac.hl7-usrpm for example. :-)

WRT using THO - 1) not until the authoring space gets better 2) US Core and C-CDA increasingly share sets, so we prefer the sets are either in THO or VSAC rather than in one IG or another

Lastly, We are working with OCL folks (OCL is all (or mostly all, and goal is to be all) FHIR based) so that maybe someday in the future, their tooling would rival VSAC authoring and it could be used in the THO space and we would not have to use VSAC at all.

@Grahame Grieve - I know you are familiar with OCL, but in case others are not: https://openconceptlab.org/

But perhaps the short term solution is using those current stewards, so publishing can get going again

this is something that has already happened - the packages are already published. There is only one short term solution, which is for simplifier to remove the size limit for at least the vsac package

even if I can restrict to active only, that's for future publications

here's the value sets that are used that are not actively maintained:

so active only doesn't seem to be a goer either

Grahame Grieve said:

so active only doesn't seem to be a goer either

OK - I thought that might be the case

Yunwei Wang so we’re stuck here. packages.fhir.org won’t handle the package. But packages2.fhir.org/packages handles it no problems, so people should always use the alternative server

Did you forget this part or did you think I abandoned it?

Ward Weistra said:

I'll continue to explore whether we can still load and serve VSAC 0.18.0+ for now.

However, I would still like to keep any size exception, if at all feasible, as small as possible. So if we could switch for any next VSAC publication from one supersize VSAC package to more tailored hl7.fhir.us.vsac.hl7-usrpm per publisher that would be a great way to not make the future impact any bigger then necessary.

So if we could switch for any next VSAC publication from one supersize VSAC package to more tailored hl7.fhir.us.vsac.hl7-usrpm per publisher that would be a great way to not make the future impact any bigger then necessary.

I'm sure not interested in that, given that then I'd be managing 19+ packages, and everyone would have to go hunting for a piece of metadata that's not easily available to decide which package they have to depend on, and you'd end up with 19 packages that occupy the same size. So what did all that achieve?

Grahame Grieve said:

packages2.fhir.org/packages

I tried http://packages2.fhir.org/packages/hl7.fhir.us.core.v700/7.0.0, which redirects to download hl7.fhir.us.core.v700#7.0.0.tgz. This tgz file is 641 bytes and the extraction contains two files: package.json and .index.json. The package contents are not in it.
image.png

no you don't want that package

http://packages2.fhir.org/packages/hl7.fhir.us.core/7.0.0,

Thanks. What's the correct way to find package url on packages2? I opened the http://packages2.fhir.org/packages/catalog and search.

and then you got a set of packages listed (http://packages2.fhir.org/packages/catalog?op=find&name=us.core&pkgcanonical=&canonical=&fhirversion=), but didn't choose the first one, which is the correct one

OK I have reported FHIR#48547 to get an explicit, sensible package size limit in place.

Also discussed the proposal we agreed on before:

• If there is a sensible size limit in place and we agree that the currently too-large-packages (VSAC) will be phased out, can we still bring in the VSAC packages that have already been used in published packages (and potentially one or more before phase out is complete)

We agree in principle to make it work, now figuring out how it can technically be done without too much impact on performance + figure out what the impact is on bandwidth/processing costs. Possibly bring it in like the tooling packages, without actually processing the contents. It won't be overnight, but we're working on it.

Grahame Grieve said:

or simplifer could simply allow bigger packages, which would be way easier for everyone

I'm really unhappy with the "you're just being difficult" framing. Let me know if I'm misunderstanding.

I think it is being underestimated how much impact this going to have on every single FHIR tool and consumer down the line. This will lead to FHIR being perceived as slow and bloated in the future if we're not vigilant.

Hence I'd also urge to find a interim solution for at least any VSAC version newer than 0.18.0 and 0.19.0. Otherwise we are still forcing 15k ValueSets and CodeSystems on anyone for even longer, while currently only <300 are being used in the whole ecosystem.

I think the CRMI operations that will return a Bundle with just the dependencies you need is the long-term solution. We can't realistically control how many artifacts will get included in a given package. That's driven by the publishing needs of whoever is putting out that specification. Inevitably, there will be a lot of content in package dependencies that isn't needed. Just as you don't want to load all of SNOMED into memory and instead just hit a terminology server to say "hey, is this code valid", it'll make sense to avoid the cost of loading large packages into memory and instead say "hey, get me the artifacts I need to process Y" of a conformance resource service and let someone else look after loading everything into memory.

I'm really unhappy with the "you're just being difficult" framing. Let me know if I'm misunderstanding.

Well, we can't change the past, so what else can I say?

we are still forcing 15k ValueSets and CodeSystems on anyone for even longer, while currently only <300 are being used in the whole ecosystem

The problem is, 'which 300'? we've found that there's no way to prospectively decide which people will want to use. I'm not interested in only doing value sets by request, where I have to issue a new VSAC every time anyone wants a value set added. Nor am I interested in splitting it up by steward, and having a whole slew of new packages to manage.

The right solution is to stop having a package altogether, but that is something that requires real $$ and no one's standing up to fund that. For now, though that might change

Hi All, we are reviewing our use of a HAPI FHIR validator built into our Git pipeline, because in order to keep it up to date requires the time of a skilled developer, and as a busy team we may not always have this resource. We were considering buying an off the shelf cloud based solution like AWS HealthLake, but we're not sure if it would meet our needs, at a quick glance it looks like it is more focused on analytics than validation. Does anyone use any FHIR validators that they can recommend, that would fit easily within our Git pipeline, requires little customization, or has a good user interface, produces human readable validation reports and can work with our terminology server. The Hammer FHIR validator looks promising, utlizing the best of both worlds, running both .NET and JAVA validators side by side, but I wasn't sure if this was still experimental. I was looking for a website that would help me make a decision on this by independantly testing the FHIR validators, listing the advantages of their features and their disadvantages, does such a site exist?

A few things to keep in mind:

• validators continue to evolve as new edge cases are identified and as the FHIR methodology evolves or is clarified. Whether you want or don't want these new issues to be caught may vary (and may vary by type of issue). So some ongoing need for configuration is likely regardless of which tool you choose
• validators typically rely on external terminology and package services and (though more stable than they once were), the interfaces for these can also change
• security considerations and threats evolve in those interfaces as well as in the languages and libraries used by all of the validators.

Taken together, it means that any tool you run locally is going to have an associated maintenance (and likely configuration) requirement. Sometimes that maintenance will have to happen in very short order.

You don’t say why the hapi validator takes a skilled developer to maintain, nor what terminology server you’re using. Nor whether you’re using the command line or you wrapped it in something

PRs are welcome, but my expectation is that any validator will require skilled maintenance.

Note that the Java FHIR validator is the most thorough validator by a long shot, and I can’t recommend to others because they don’t pass the test cases. The test cases are unfriendly so the mere fact they don’t pass isn’t necessarily a problem but I do know the others are less thorough

Hi @Grahame Grieve and @Lloyd McKenzie thanks for replying. It's more of a question that we are small team of public sector employees, busy working on FHIR interoperability solutions, and updating our existing customized HAPI FHIR validator that is built into our Git pipeline (in future we were hoping to tie this to our NHS Terminology Server (Customised version of CSIRO Ontoserver technology), takes a certain amount of effort. That's why we wondered if there was an off the shelf solution that would pass the test cases mentioned make validation easier, it doesn't sound like there is, as the validator would need to be customised to our needs, of being valid against the latest or appropriate version of the UK Core, and valid against the SNOMED CT terminology we use.

I was listening to @Vadim Peretokin excellent presentation on FHIR validation at past Dev Days event, and I think he says pretty much the same as you, that the Java validator is the most battle tested validator in the FHIR eco system. @Vadim Peretokin can we use the Hammer FHIR validator in a git pipeline? Currently, we have a FHIR Validator that runs at least once a day, and will check for changes when pull requests are made.

Grahame Grieve said:

You don’t say why the hapi validator takes a skilled developer to maintain, nor what terminology server you’re using. Nor whether you’re using the command line or you wrapped it in something

PRs are welcome, but my expectation is that any validator will require skilled maintenance.

Note that the Java FHIR validator is the most thorough validator by a long shot, and I can’t recommend to others because they don’t pass the test cases. The test cases are unfriendly so the mere fact they don’t pass isn’t necessarily a problem but I do know the others are less thorough

as the validator would need to be customised to our needs, of being valid against the latest or appropriate version of the UK Core, and valid against the SNOMED CT terminology we use.

neither of those things should require a customised validator, unless UK core isn't conformant with FHIR itself, in which case you've got a huge problem irrespective of which validator you use

since hammer is a wrapper around the validator, I don't understand what it gets you in a pipeline like that?

@Grahame Grieve the UK Core is conformant with the FHIR standard, but as an example, where we have constrained the Patient.identifier to use an extension, NHSNumberVerificationStatus, we would expect to see for example, where in an instance example of the where if the NHS Number is present and verified, then we would expect a validator to check that the extension is present and the code/ display value of "01" / "Number present and verified" is present in that instance example. That is the kind of custom validation that I am referring to that goes beyond the capabilities of an "out of the box" FHIR validator. For example, if someone had made a mistake in the instance example and used "02"/ "NHS Number present and verified", instead of "01" / "Number present and verified".

Regarding Hammer, I only heard about it this week, and I need to do some background reading to understand it's full capability full potential. If anyone can point to any documentation or sites, that can give advice on FHIR validators, feel free to comment.

Grahame Grieve said:

neither of those things should require a customised validator, unless UK core isn't conformant with FHIR itself, in which case you've got a huge problem irrespective of which validator you use

the out of the box validator will validate the proper extension if the profiles you are using declare them and have constraints like you mentioned.

ok sure, a validator isn't going to enforce business logic like that that's not expressed anywhere, though such logic can usually be expressed using FHIRPath, and then will be enforced

Hi @Jean Duteau where you mention about profiles declare an extension, is that the same as define an extension: https://www.hl7.org/fhir/defining-extensions.html ? For example , use a canonical URL that uniquely identifies the extension, specify it's context, set it's cardinality, publish and reference the extension's canonical URL that uniquely identifies the extension in the profile. We do that, it's the business rules, alongside the standard FHIR rules that we need a validator to work on,

Jean Duteau said:

the out of the box validator will validate the proper extension if the profiles you are using declare them and have constraints like you mentioned.

There's two steps:

1. Define the extension as a StructureDefinition and publish it in a package that's made available to the validator
2. Define a profile on the resource you want to reference the extension and add a slice for that extension saying that the extension is required (cardinality 1..1) and adding any constraints that define other rules.

@John George would you be using the validator during ig development in the ci-build? because as far as I see you have described the required rules already in https://simplifier.net/guide/uk-core-implementation-guide-stu2/Home/ProfilesandExtensions/Profile-UKCore-Patient?version=2.0.1 and could use that package to validate any examples by specifing the needed profile. if you could share your git pipeline requirements, that would be helpful.

The original post mentioned a desire to have a validator that "has a good user interface, produces human readable validation reports" - apart from a few web frontends for $validate, and Simplifier and Hammer, there are not a lot of options. "Human readable validation reports" - I don't think any of the tools support that (on the assumption that one could create human readable reports for deep validation).

John George said:

in future we were hoping to tie this to our NHS Terminology Server (Customised version of CSIRO Ontoserver technology)

Let me know what help you need and I'll make sure you get it (there isn't anything customised about the Ontoserver being used for the NHS Terminology Server so anything you can do with Ontoserver, you'll be able to do with the NHS Terminology Server)

"Human readable validation reports"

what's a human readable validation report? The java validator has various output formats, one of which is html, which everyone is used to looking at in the IG publisher wrapper

But otherwise, what? It seems like not a big lift to add something to the validator for an output of what is desired, and there's already a framework for multiple outputs, so that exists

the other challenge which I'm always sweating on is how to make the messages more comprehendible

I'd estimate that "99% of implementers don't use IG Publisher". Authoring an IG is only done by a very small % of the community, and one can't assume FHIR implementers to be familiar with it.

When it comes to validation, you'd want both a very precise indication where in a resource the error occurs, with a comprehensible error message for someone not terribly well versed in FHIR and strucDefs.

ok, not everyone, that's true

with a comprehensible error message for someone not terribly well versed in FHIR and strucDefs.

I've given up on that. I don't know how to explain stuff. I mean, I try, but the language is rooted in the FHIR definitions, and I don't expect much from non-FHIR developers

Thanks @René Spronk for your input, yes we want a validator that not only my team at NHS England can understand the error messages, but possibly in future one that can be widely used throughout the NHS, so implementors of our FHIR specification, who may not be so well versed in FHIR can easily understand an error message and pinpoint where exactly the problem is. Having many years ago, worked in a hospital on pathology messaging, I can relate to this, and it would have been useful to have a validator who messages were easy to understand, rather than escalating this to our IT supplier. I want to find out if the HAPI FHIR validator that we use along side the .NET FHIR validator in Simplifier.net is sufficient, or if there have been any recent developments in FHIR validators that may mean there are better FHIR validation solutions out there.

René Spronk said:

When it comes to validation, you'd want both a very precise indication where in a resource the error occurs, with a comprehensible error message for someone not terribly well versed in FHIR and strucDefs.

suggestions to improve the error messages are always welcome

To help to understand validation issues we are supporting a student project at the Bern University of Applied Sciences, who are trying to determine with the help of a LLM what the underlying problem is based on the error messages from the Java validator/matchbox and how it could be rectified. The background is that there can be a lot of follow up warnings/errors with specific FHIR documents due to the slicing, and we want to find out whether LLM can make a direct recommendation on what needs to be corrected.

slicing is particularly difficult, yes

Despite best efforts, messages will never be perfectly easy to understand, by everyone.
As well as making them as simple as is practical, perhaps a link to an online FAQ could be given, which could spell out some common explanations (e.g. what a slice is). Also the FAQ can give a link to Zulip, as a last recourse. We don't want to replace an automated tool with humans, but, otoh, it is always good to get people into the community.

well, Rik, there's only 1152 messages that validator can produce :grinning:

Grahame Grieve said:

suggestions to improve the error messages are always welcome

For grouping of bulk FHIR Validation results like we do by https://git.uni-greifswald.de/CURDM/Bulk-FHIR-Validation/src/branch/main/README.md IMHO it would help to have (the main part of) error messages additionally/structured (maybe extension) without the used code / only on codesystem level, since in some cases while bulk validation too many different but roughly the same error messages (for each used code of a codesystem if many different values like ICD codes from many validated resources) which i sometimes group for a FHIR element to one single/aggregated error for the whole code system (at the moment by removing the code from message by regex, which could fail if the validators messages/output format will change).

that actually sounds like a suggestion to not change the messages!

maybe giving you message id will make it easier? But how can I remove the code from the message? That doesn't sound like a useful thing to do

To change messages to improve / to understand them better is very good. To remove the code genarally would be very bad (it helps and i want to see it very often). :)

Just wanted to mention, it would be good to be able to not use it/remove it for some (not all) further analysis.

So if no additional/separate $validate operation outcome element for the message without the code available, it would help to have as possible stable output patterns to be able detect/separate/extract the codesystem and code parts of the output string.

Another thing: It could help some users without deep understanding of validator internas, if he/she could distinguish if an error occurs because of i.e. validator config/external profiles (f.e. if a valueset could not be expanded) or the error is in the validated resource.

The core problem is our ontology service requires authorisation to be performed.

So we wrote an interceptor to handle this.
It also has some code to handle packages held on simplifier, not sure if that is needed anymore.

Other than that it is just the Java FHIR validator

The core problem is our ontology service requires authorisation to be performed.

You could make a PR to the core - others are interested in this. Though I'm not sure that'd we'd accept it - need a good test case

It also has some code to handle packages held on simplifier, not sure if that is needed anymore.

no not needed

Agree.

We also use this as a facade/application service to the to server (and AWS FHIRWorks) for FHIR apps like NLM Form builder

Hi @Oliver Egger our main validator is the IOPS GitHub CI CD Test Script , it validates all FHIR resources in a GitHub repo on commit. Fails on warnings (by design) and runs automatically on commit. We use it once a team member feels FHIR resources are ready to be added to the repo/simplifier (we would expect before this the team member would have done some validation in Forge/Simplifier (. NET) before committing FHIR resources to the repository. It's used on any repositories produced by my team on GitHub (Actions tab).  Runs automatically on commit.

The details are:

GitHub Action IOPS-FHIR-Validation-Terminology Workflow 
Push from FHIR repo e.g. FHIR-R4-UKCORE-STAGING-MAIN with ontoserver credentials 

Check out IOPS-Validation 
Adds IOPS-FHIR-Test-Scripts as folder inside FHIR-R4-UKCORE-STAGING-MAIN folder in the local ubuntu machine. 

Check out validation-service-fhir-r4 
Adds IOPS-FHIR-Validation-Service as folder inside FHIR-R4-UKCORE-STAGING-MAIN folder in the local ubuntu machine. 

Install npm 
Install npm in IOPS-FHIR-Test-Scripts folder 

Configure FHIR Validator 
npm start in IOPS-FHIR-Test-Scripts folder (configures FHIR validator using ontoserver credentials). The start is defined in package.json as "ts-node src/configureValidator.ts" 

Build FHIR Validator 
Runs mvn clean install inside IOPS-FHIR-Test-Scripts 
Clean: remove target folder 
Package: Follows the lifecycle phase 
validate >> compile >> test (optional) >> package 
(for reference install: validate >> compile >> test (optional) >> package >> verify >> install) 

Run FHIR Validator 
          nohup java -jar validation-service-fhir-r4/target/fhir-validator.jar --terminology.url=https://ontology.nhs.uk/production1/fhir --terminology.authorization.tokenUrl=https://ontology.nhs.uk/authorisation/auth/realms/nhs-digital-terminology/protocol/openid-connect/token --terminology.authorization.clientId=${{ secrets.ONTO_CLIENT_ID }} --terminology.authorization.clientSecret=${{ secrets.ONTO_CLIENT_SECRET }} --aws.validationSupport=false --aws.queueEnabled=false & sleep 120 
nohup: no hang up is a command in Linux systems that keep processes running even after exiting the shell or terminal 
java -jar: run the jar file passing the parameters listed after each --  

Run Test 
Run npm test inside IOPS-FHIR-Test-Scripts folder. The test is defined in package.json as  
jest --runInBand src/validate.test.ts

Oliver Egger said:

John George would you be using the validator during ig development in the ci-build? because as far as I see you have described the required rules already in https://simplifier.net/guide/uk-core-implementation-guide-stu2/Home/ProfilesandExtensions/Profile-UKCore-Patient?version=2.0.1 and could use that package to validate any examples by specifing the needed profile. if you could share your git pipeline requirements, that would be helpful.

Kevin Mayfield said:

The core problem is our ontology service requires authorisation to be performed.

There is potentially a simple decoupled approach for this (using a locked down authenticating proxy) that would avoid needing to change the Validator ... just need to make sure that the content licensing requirements are sufficiently met in the context of use

@John George it sounds like your developers should be running a complete validation before committing. It is possible to use the java command line validator on each developers machine, before any git pipeline based validation. I use it from a button in my IDE (Oxygen). It completely validates the current file against whatever set of profiles I have configured.

Hi @Rik Smithies, thanks for your suggestion of using the Java command line validator. Perhaps I oversimplified the process, I've updated my response to @Oliver Egger's question about our git pipeline. Personally, before committing an instance example to the Git Repo I, aside from checking it's valid in Simplifier, I will run it past the ## FHIR Development and Testing (FHIR Validation) Skunkworks product <http://lb-fhir-validator-924628614.eu-west-2.elb.amazonaws.com/swagger-ui/index.html#/>. It's good for debugging and fault finding.

Rik Smithies said:

John George it sounds like your developers should be running a complete validation before committing. It is possible to use the java command line validator on each developers machine, before any git pipeline based validation. I use it from a button in my IDE (Oxygen). It completely validates the current file against whatever set of profiles I have configured.

@Rik Smithies I like the sound of an FAQ that could decipher an error message. For example, I have only learnt from experience of validating instance example that an error message like "is not a correct literal for a URI", means in layman's terms there is a typo in the URI, probably an extra . or / or - for example, but this might not be obvious to everyone.

Rik Smithies said:

Despite best efforts, messages will never be perfectly easy to understand, by everyone.
As well as making them as simple as is practical, perhaps a link to an online FAQ could be given, which could spell out some common explanations (e.g. what a slice is). Also the FAQ can give a link to Zulip, as a last recourse. We don't want to replace an automated tool with humans, but, otoh, it is always good to get people into the community.

@John George just to be clear, the java command line validator, is the same engine as the HAPI one that you were suggesting to integrate. I am all for "belt and braces" but if you can validate with HAPI before checking into git then you may not have the same need to integrate it into the pipeline. It doesn't really need a "skilled developer" to use, or update, the command line version (though there is still the somewhat separate "messages" issue). Once you know the command line (and maybe put it into a batch file that anyone can use) then updating it is a simple matter of downloading it and saving a new .exe into place. Also the IG publisher will ultimately validate your instances again, so that is yet another check.

The github action is effectively automating most of what you've said.

It's not a validator like was said earlier but just a CI/CD test script which uses a validator.