[Gmsh] Hdf5 files and gmsh

[paul francedixhuit]

 Hi All


Recently I found interesting discussions about hdf5 and gmsh (see
https://gitlab.onelab.info/gmsh/gmsh/issues/552). On my side I had to look
to xmdf format 2 years ago that seemed to be interesting to my applications
(mechanical engineering). Finally I gave up because of difficulties and
xdmf developments not really active.


I'm using hdf5 files through h5py Python library to deal with huge amount
of data, in either static conditions or transitory ones.


In the meantime gmsh team has developed the format 4.1 and I've to dig into
it;  I saw about Python API and I'm not familiar with, but not sure that's
I'm looking for. In my opinion it'll be very useful and powerful to be able
to import datasets from hdf5 files directly from the gmsh *native* file,
especially for post-processing purposes; let me developing:

- by import I mean calling, not intending to write it in an ascii format;
similar to "h5.get" for people who are familiar to h5py library

- I'm not speaking about HPC modelings where huge meshes and models can be
post-processed using dozens or hundreds of cores, but about a common PC or
a working station having limited resources compared to HPC servers,

- When working on models having a million of 2nd order elements, and dozens
of time steps, dealing with different variables (up to 3 displacement
components, up to 6 for the stress ones, and so on), *all results cannot be
loaded in a single native .pos file* due to obvious computing limitations


While using hdf5 in native gmsh file, I've been thinking it may answer to
this because:

- *hdf5 files are optimized for I/O* in terms of speed and memory; for
example only datasets you need are loaded and not the entire file: it's
fast accordingly

- data can be compressed in hdf5 file

- a dataset is a block of data (typically a matrix)

- in practice in gmsh, only some data are plotted at the same time and not
all: for example not relevant to plot Von Mises stress and equivalent
strain at the same time, or as well initial time step and final one

- in other word, only what you need is "uploaded" through something like
import "myhdf5\dataset129" (is slicing will be possible?)

- each dataset is built following gmsh format

- *as many import as necessary* will be done due to different dataset
sizes: one per element type (triangles, quads, tets, hex, wedges, and so on)

- 1 dataset per element = elements ordering might be a limitation I think
(speed down), except if it can be anticipated

- if slicing not possible, well data must be duplicated (higher file size),
but fast reading in practice

- and so on


I've been using hdf5 files for some time, and it's now an essential part of
my data management: I'm sure gmsh will take huge advantages to integrate it.


I'll be happy to discuss further about it


Thanks to gmsh team for their works


Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://onelab.info/pipermail/gmsh/attachments/20200322/9f9c8f28/attachment.html>