[Getdp] Solved: Compiling parallel (MPI) version of Petsc for 64 bit Windows with GNU compilers

[michael.asam at infineon.com]

Hi all,

after a long time I've finally succeeded in compiling a 64 bit parallel version of Petsc for Windows.
All tries with MPICH2 failed. It is known to have problems with newer 64 bit Windows (e.g. with UAC).
Therefore I gave Microsoft's MPI (which is also free of charge) a try.
Unfortunately there are some problems when using mingw compilers. (Some Fortran symbol
names are not compatible). So it does not work out of the box.
But the problems can easily be solved.

Here is the way it works:

Preparing Microsoft's MPI to work with x86_64-w64-mingw32-gfortran

0.) You need Cygwin. Please make sure that the Devel tools and Python are installed.

1.) Download Microsoft's "HPC Pack 2008 R2 MS-MPI Redistributable Package":
http://www.microsoft.com/en-us/download/details.aspx?id=14737

2.) Install it in a Directory without any spaces in the path/folder name

3.) Insert the following line at the beginning of the file InstallationPath\Inc\mpi.h
#include <stdint.h>

4.) In the file InstallationPath\Inc\mpif.h
replace INT_PTR_KIND()
by 8

5.) Copy C:\Windows\System32\msmpi.dll to InstallationPath\Lib\amd64\msmpi.dll

6.) In a Cygwin terminal please enter the following:
cd /cygdrive/c/your/HPC/installation/path/Lib/amd64
gendef msmpi.dll
x86_64-w64-mingw32-dlltool -d msmpi.def -l libmsmpi.a -D msmpi.dll

This creates a library (libmsmpi.a) which now works with x86_64-w64-mingw32-gfortran.


Compiling Petsc

7.) Download Petsc v3.1-p8: petsc-3.1-p8.tar.gz<http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-3.1-p8.tar.gz>  (or http://www.mcs.anl.gov/petsc/download/index.html)

8.) Untar it (In a Cygwin terminal go to the folder where you've downloaded petsc-3.1-p8.tar.gz<http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-3.1-p8.tar.gz>
and type: tar -xzvf petsc-3.1-p8.tar.gz) and enter this directory (cd petsc-3.1-p8).

9.) Edit the file src/sys/memory/mal.c:
Change in line 39 (unsigned long) to (unsigned long long)

10.) Edit the file include/mpiuni/mpi.h:
Change in line 112
#define MPIUNI_INTPTR long
to:
#define MPIUNI_INTPTR long long

11.) Set the PATH variable in the Cygwin terminal:
export PATH=/usr/x86_64-w64-mingw32/bin:/usr/x86_64-w64-mingw32/sys-root/mingw/bin:/usr/local/bin:/usr/bin:/bin: / cygdrive/c/your/HPC/installation/path/Bin:$PATH

12.) Configure PETSc with the command given below in the Cygwin shell. Please adapt your mpi path and choose with-scalar-type=complex
or with-scalar-type=real according your needs.
Please note: GetDP can solve problems with real or complex DOFs regardless the scalar-type of Petsc. In this context the scalar-type is just an optimization in terms of speed and memory consumption.

./configure --CC=x86_64-w64-mingw32-gcc.exe --CXX=x86_64-w64-mingw32-g++.exe --FC="x86_64-w64-mingw32-gfortran.exe  -fno-range-check" --CPP=x86_64-w64-mingw32-cpp.exe --with-debugging=0 --with-clanguage=cxx --with-shared=0 --with-x=0 --useThreads=0 --download-f-blas-lapac=ifneeded --download-mumps=ifneeded   --download-parmetis=ifneeded   --download-scalapack=ifneeded  --download-blacs=ifneeded  --with-scalar-type=complex --with-mpi-include=/cygdrive/c/your/HPC/installation/path/Inc --with-mpi-lib=/cygdrive/c/your/HPC/installation/path /Lib/amd64/libmsmpi.a

13.) Before Petsc is compiled the file ./cygwin-cxx-opt/include/petscconf.h  has to be edited.
Replace ( BEWARE: check the number of underscores, don't touch to PETSC_HAVE__SLEEP )

#ifndef PETSC_HAVE_SLEEP
#define PETSC_HAVE_SLEEP 1
#endif

with

#undef PETSC_HAVE_SLEEP

and replace

#ifndef PETSC_HAVE_GETPAGESIZE
#define PETSC_HAVE_GETPAGESIZE 1
#endif

with

#undef PETSC_HAVE_GETPAGESIZE


14.) Now let's compile it.
make PETSC_DIR=/your/path/to/petsc-3.1-p8 PETSC_ARCH=cygwin-cxx-opt all

15.) Drink some coffee and pray that everything works ;-)

That's all.

Hopefully this is helpful for some of you.


Have a nice day!
Michael




Michael Asam
Infineon Technologies AG
ATV BP PD1 M1, Munich

Infineon Technologies AG
Vorsitzender des Aufsichtsrats: Wolfgang Mayrhuber
Vorstand: Peter Bauer (Vorsitzender), Dominik Asam, Arunjai Mittal, Dr. Reinhard Ploss
Sitz der Gesellschaft: Neubiberg
Registergericht: München HRB 126492