<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
Hello again Helmut,<br>
<br>
I had issues similar to yours when I started with getdp... UMFPACK
often crashed on my machines at some load thresold. I don't really
have an explanation, sorry.<br>
Then, I compiled getdp against petsc with MUMPS. It works really
well, even with problems as big as millions of DOF requiring tens of
gigs of memory (then it's better if you build the parallel version,
though), as reported by the developers.<br>
The time spent on the compilation of petsc/getdp really worth it,
IMO...<br>
Here is how I compiled my last getdp:<br>
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
</w:Compatibility>
<w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel>
</w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" LatentStyleCount="156">
</w:LatentStyles>
</xml><![endif]--><!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.0pt;
font-family:"Times New Roman";
mso-ansi-language:#0400;
mso-fareast-language:#0400;
mso-bidi-language:#0400;}
</style>
<![endif]-->
<p class="MsoNormal" style=""><span style="font-size: 10pt;
font-family: Courier;">export
PETSC_DIR=yourPETCdir
PETSC_ARCH=yourPETSCarch<br>
./configure \<br>
--with-cc=</span><span style="font-size: 10pt; font-family:
Courier;">/yourMPIpath/bin</span><span style="font-size: 8pt;
font-family: "Courier New"; color: red;">/</span><span
style="font-size: 10pt; font-family: Courier;">mpicc </span><span
style="font-size: 10pt; font-family: Courier;">\<br>
--with-fc</span><span style="font-size: 10pt; font-family:
Courier;">=/</span><span style="font-size: 10pt; font-family:
Courier;">your</span><span style="font-size: 10pt; font-family:
Courier;">MPIpath/</span><span style="font-size: 10pt;
font-family: Courier;">bin</span><span style="font-size: 8pt;
font-family: "Courier New"; color: red;">/</span><span
style="font-size: 10pt; font-family: Courier; color: red;">mpif90
</span><span style="font-size: 10pt; font-family: Courier;">\<br>
--with-f77=</span><span style="font-size: 10pt; font-family:
Courier;">/</span><span style="font-size: 10pt; font-family:
Courier;">your</span><span style="font-size: 10pt; font-family:
Courier;">MPIpath</span><span style="font-size: 8pt;
font-family: "Courier New"; color: red;">/</span><span
style="font-size: 10pt; font-family: Courier;">mpif77 \<br>
</span><span style="font-size: 10pt; font-family: Courier;">--with-cxx=</span><span
style="font-size: 10pt; font-family: Courier;">/</span><span
style="font-size: 10pt; font-family: Courier;">your</span><span
style="font-size: 10pt; font-family: Courier;">MPIpath</span><span
style="font-size: 8pt; font-family: "Courier New";
color: red;">/</span><span style="font-size: 10pt; font-family:
Courier;">mpicxx </span><span style="font-size: 10pt;
font-family: Courier;">\<br>
</span><span style="font-size: 10pt; font-family: Courier;">--with-petsc-prefix=$PETSC_DIR
\<br>
--with-petsc-arch=$PETSC_ARCH
\<br>
--with-gsl-prefix=$GSL_BASE</span><span style="font-size: 10pt;
font-family: Courier;"><br>
--enable-petsc \<br>
</span><span style="font-size: 10pt; font-family: Courier;">--enable-slepc
\<br>
</span><span style="font-size: 10pt; font-family: Courier;">--disable-sparskit
\<br>
</span><span style="font-size: 10pt; font-family: Courier;">--disable-arpack
\<br>
</span></p>
Hope this helps,<br>
<br>
Guillaume<br>
<br>
<br>
On 14/12/2010 09:31, Helmut Müller wrote:
<blockquote
cite="mid:B96C7AFC-F068-4067-A8FC-09EAF83B557D@ingenieurbuero-mbi.de"
type="cite">
<div>Hello Guillaume,</div>
<div><br>
</div>
<div>thank you for your reply.</div>
<div><br>
</div>
<div>Yes, problems with 200000 DOF can be solved on this machine,
without probs. But larger problems with > 250.000 DOF always
lead to malloc failure messages. I assume not the machine is
limiting the size of solvable probs, but getdp, at least in the
precompiled dl version.</div>
<div>Unfortunately I didn´t manage to recompile getdp by myself. I
have had different tries in the past, but no success, so I use
the downloadable version. </div>
<div><br>
</div>
<div>Perhaps I´ll have to dig myself into recompilation again. Can
you confirm that a recompiled version would solve larger
problems than the precompiled version? Next question :-) why ?
And then, why ist the precompiled Version limited ?</div>
<div><br>
</div>
<div>Please, do not missunderstand me, I don´t want to complain
about this software or the service that precompiled Versions are
availlable, but I´d like to understand ( typical engeneer ).</div>
<div><br>
</div>
<div>Thanks in advance, best regards,</div>
<div><br>
</div>
<div>Helmut Müller</div>
<div><br>
</div>
<div>PS: just tried your suggested command with getdp version
2.1.0 :</div>
<div><br>
</div>
<div>
<div>Info : InitSolution[T]</div>
<div>Info : Generate[T]</div>
<div>Info : (CPU = 13.1382s, Mem = 537Mb) </div>
<div>Info : Solve[T]</div>
<div>Info : N: 225215</div>
<div>getdp(19601) malloc: *** mmap(size=2147483648) failed
(error code=12)</div>
<div>*** error: can't allocate region</div>
<div>*** set a breakpoint in malloc_error_break to debug</div>
<div><br>
</div>
</div>
<div>( this message repeated several times)</div>
<div><br>
</div>
<div>getdp(19601,0xa052c500) malloc: *** mmap(size=1948217344)
failed (error code=12)</div>
<div>
<div>*** error: can't allocate region</div>
<div>*** set a breakpoint in malloc_error_break to debug</div>
<div><br>
</div>
<div>UMFPACK V5.2.0 (Nov 1, 2007): ERROR: out of memory</div>
<div><br>
</div>
<div>[0]PETSC ERROR: --------------------- Error Message
------------------------------------</div>
<div>[0]PETSC ERROR: Error in external library!</div>
<div>[0]PETSC ERROR: umfpack_UMF_numeric failed!</div>
<div>[0]PETSC ERROR:
------------------------------------------------------------------------</div>
<div>[0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 7, Mon
Jul 6 11:33:34 CDT 2009</div>
<div>[0]PETSC ERROR: See docs/changes/index.html for recent
updates.</div>
<div>[0]PETSC ERROR: See docs/faq.html for hints about trouble
shooting.</div>
<div>[0]PETSC ERROR: See docs/index.html for manual pages.</div>
<div>[0]PETSC ERROR:
------------------------------------------------------------------------</div>
<div>[0]PETSC ERROR: problem.pro on a umfpack-c named
MacBookPro.local by helmutmuller Tue Dec 14 15:26:14 2010</div>
<div>[0]PETSC ERROR: Libraries linked from
/Users/geuzaine/src/petsc-3.0.0-p7/umfpack-cxx-opt/lib</div>
<div>[0]PETSC ERROR: Configure run at Tue Aug 18 10:37:03 2009</div>
<div>[0]PETSC ERROR: Configure options --with-debugging=0
--with-scalar-type=complex --with-clanguage=cxx
--with-shared=0 --with-x=0 --with-fortran=0 --with-mpi=0
--with-umfpack=1 --download-umfpack=ifneeded</div>
<div>[0]PETSC ERROR:
------------------------------------------------------------------------</div>
<div>[0]PETSC ERROR: MatLUFactorNumeric_UMFPACK() line 176 in
src/mat/impls/aij/seq/umfpack/umfpack.c</div>
<div>[0]PETSC ERROR: MatLUFactorNumeric() line 2390 in
src/mat/interface/matrix.c</div>
<div>[0]PETSC ERROR: PCSetUp_LU() line 222 in
src/ksp/pc/impls/factor/lu/lu.c</div>
<div>[0]PETSC ERROR: PCSetUp() line 794 in
src/ksp/pc/interface/precon.c</div>
<div>[0]PETSC ERROR: KSPSetUp() line 237 in
src/ksp/ksp/interface/itfunc.c</div>
<div>[0]PETSC ERROR: KSPSolve() line 353 in
src/ksp/ksp/interface/itfunc.c</div>
<div>[0]PETSC ERROR: User provided function() line 62 in
unknowndirectory/LinAlg_PETSC.cpp</div>
<div>Abort trap</div>
</div>
<div><br>
</div>
<div>So I was unable to solve a problem wit 225215 DOF. So, how
the hell have you ( or Christophe) been able to solve about 10
millions DOF ?</div>
<div><br>
</div>
<div>regards, Helmut</div>
<div><br>
</div>
<div><br>
</div>
<br>
<div>
<div>Am 14.12.2010 um 14:31 schrieb Guillaume Demesy:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div text="#000000" bgcolor="#ffffff"> Hello Helmut,<br>
<br>
8Gb should be more than enough to solve this 200000 DOF
problem.<br>
Getdp binaries are compiled with the PETSc solver umfpack.
Have you tried it? Or are you using GMRES?<br>
getdp myfile.pro -solve myresolution -ksp_type preonly
-pc_type lu -pc_factor_mat_solver_package umfpack <br>
<br>
But the best solution when tackling big problems is probably
to <br>
1- recompile your own petsc with openmpi support and MUMPS<br>
2- compile getdp with your petsc<br>
...which will provide you a parallel 'solve'. The
preprocessing will remain serial.<br>
<br>
see:<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://geuz.org/trac/getdp/wiki/PETScCompile">https://geuz.org/trac/getdp/wiki/PETScCompile</a>
<br>
./configure --CC=/opt/local/bin/openmpicc
--CXX=/opt/local/bin/openmpicxx
--FC=/opt/local/bin/openmpif90 --with-debugging=0
--with-clanguage=cxx --with-shared=0 --with-x=0
--download-mumps=1--download-parmetis=1--download-scalapack=1
--download-blacs=1 --with-scalar-type=complex<br>
<br>
Good luck!<br>
<br>
Guillaume<br>
<br>
<br>
<br>
On 14/12/2010 06:28, Helmut Müller wrote:
<blockquote
cite="mid:25315A0F-1401-4985-BBB6-C61B591BA648@ingenieurbuero-mbi.de"
type="cite"><span class="Apple-style-span"
style="font-family: Times;"> </span><span
class="Apple-style-span" style="font-family: Times;">
<pre>Hi all,</pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;"></span><span
class="Apple-style-span" style="font-family: Times;"> </span><span
class="Apple-style-span" style="font-family: Times;">
<pre>first I´d like to thank you for this impressive software!</pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;"></span><span
class="Apple-style-span" style="font-family: Times;"> </span><span
class="Apple-style-span" style="font-family: Times;">
<pre>I use it for (quite simple) simulations regarding buildingphysics, I just solve heat equations. Therefore I have quite large models (2m by 2m by 2m) with some small parts or thin layers (10mm).</pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;">
<pre>Unfortunately I have to spend a lot of time adjusting the mesh and/or simplify the geometry because I didn´t manage to solve Problems with more than approx. 220.000 DOF on my Mac (8GB RAM, quadCore ). Those problems are solved within seconds or few minutes.</pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;"></span><span
class="Apple-style-span" style="font-family: Times;"> </span><span
class="Apple-style-span" style="font-family: Times;">
<pre>From working with other FEM Simulations I know that it is really important to have a "good" mesh, but I´d like to spend less time for optimization of the geometry and/or the mesh for the price of longer calc times on larger models. A longer calculation time </pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;">
<pre>would cost me far less than optimization. </pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;"></span><span
class="Apple-style-span" style="font-family: Times;"> </span><span
class="Apple-style-span" style="font-family: Times;">
<pre>In this context I have read a mail on the list:</pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;">
<pre>> This has been successfully tested with both iterative (GMRES+SOR) and </pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;">
<pre>> direct (MUMPS) parallel solvers on up to 128 CPUs, for test-cases up to </pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;">
<pre>> 10 millions dofs.</pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;"></span><span
class="Apple-style-span" style="font-family: Times;"> </span><span
class="Apple-style-span" style="font-family: Times;">
<pre>With which trick or procedure has this been done ? On which Platform ? How can I at least use the available memory to perform such calculations ( my getdp 2.1 on MacOS (binary download) uses only a small part ca. 1GB of the available memory, pets fails with </pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;">
<pre>a malloc error message, the new release of getdp uses all cores but with no benefit for maximum possible model size in respect to DOF. So I assume with 8GB it should be possible to do calculations of at least 500000 DOF.</pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;"></span><span
class="Apple-style-span" style="font-family: Times;"> </span><span
class="Apple-style-span" style="font-family: Times;">
<pre>So, what do I miss ? Could partitioning of the mesh and doing separate, iterative calculations be a solution ?</pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;"></span><span
class="Apple-style-span" style="font-family: Times;"> </span><span
class="Apple-style-span" style="font-family: Times;">
<pre>Thanks in advance for any suggestion. I assume that other people are interested in this too.</pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span><span class="Apple-style-span"
style="font-family: Times;"></span><span
class="Apple-style-span" style="font-family: Times;"> </span><span
class="Apple-style-span" style="font-family: Times;">
<pre>Helmut Müller</pre>
</span><span class="Apple-style-span" style="font-family:
Times;"> </span>
<div><br>
</div>
<pre wrap=""><fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
getdp mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:getdp@geuz.org">getdp@geuz.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.geuz.org/mailman/listinfo/getdp">http://www.geuz.org/mailman/listinfo/getdp</a>
</pre>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
</body>
</html>