Data formats review

This topic is to organize the next in our short series of meetings on neuroimaging data formats.

At the last meeting, we discussed the ADSF paper, and thought about desirable properties of a new data format.

At the end of the meeting, we agreed that it would be useful to discuss the various data formats in common use for analysis at the moment, with their benefits and dysbenefits. Let’s say those are:

  • NIfTI (Chris Rorden)
  • MINC
  • AFNI
  • [edit] DICOM

We said it would be good to have a short talk about each, followed by discussion.

The first question is - who would like to talk about these formats?

I guess we’ll end up with a meeting in a few weeks, if the stars align.

2 Likes

For the presentations, I suggest we do something like:

  • What is the format?
  • Who’s using it?
  • What is it really good at?
  • What do you miss when you use it?
  • If you were making a 2.0 version of the format, what would change?

I would be happy to chat about NIfTI. I have some draft slides on Google Drive. If anyone wishes to extend or amend my comments, please contact me and I can provide editorial permissions.

@matthew.brett I would suggest you add DICOM to the list of formats to discuss. I would suggest Andrey Fedorov could present this, and perhaps describe his dcmqi (DICOM ( dcm ) for Quantitative Imaging ( qi )).

Chris - great - thanks for volunteering for NIfTI, that’s very helpful.

I did think of DICOM, but it seems to me that its currently very rare to use that as an analysis format - do you agree?

@matthew.brett I agree that DICOM analyses are very rare now. However, if our remit is to discuss future formats, it seems an opportunity to discuss whether the dominant format used clinically could fill this niche. One could leverage a huge amount of existing infrastructure. It is not only a format, but a proven method for storage, query and transfer.

It is true that DICOM is historically very complex, with many compression strategies that are not seen elsewhere. It has been interpreted differently by different vendors. Further, vendors have introduced errors over the years, so robust tools need to include kludges for these. Classic DICOM saves each 2D image as a separate file, which can overwhelm file systems.

On the other hand, Canon and Siemens (XA) have shown how modern enhanced DICOM can save an entire series in a lean single file. Since DICOM only requires compliant tools to handle the uncompressed transfer syntax, one can envision a future DICOM that is easy to support robustly.

If our remit is to envision future formats, I would advocate that we seriously consider if cooperation with WG-16 could benefit both clinical and research communities. From my perspective, the last couple decades have seen a growing divergence between the research and clinical communities. I would argue we should seek a virtuous cycle that would allow much easier translation of research tools to change the standard of care.

1 Like

I share @neurolabusc 's perspective on this. DICOM is far from perfect and is often frustrating, but that can be said of other formats too if we’re honest. And DICOM will always be part of the neuroimaging discussion because it is what will come out of the scanners for the foreseeable future. I’d be interested in leveraging things like zarr and ngff for programming convenience and performance without losing the standardized representation of acquisition parameters, subject ids, demographics, etc.

Just to clarify - I was thinking specifically of gathering experience of using these formats in analysis - rather than reviewing potential successor formats - which I suppose could include the new DICOM. I’d prefer to stick to discussions of the format itself rather than particular software to support it - although that will obviously be tough for AFNI.

But - having said that - I can see the argument for including DICOM, in reviewing what it is good at and what it is bad at.

So let’s add it to the list - I’ve edited the original email above - and I’ve taken the liberty of signing you up (Chris) for Nifti.

I believe we now have kind volunteers for all 4 formats.

I’ve made a WhenIsGood link to decide when to meet:

Please do fill in when you can.