A colleague here helped me recompile getdp using sparskit. I am still within a factor of 2 of the hard (32-bit imposed) memory limit, but at least this version is working on my immediate problem, since it requires less than 1 GB. Are there any plans for a 64-bit version?<br>
<br>--John<br><br><br><div class="gmail_quote">On Sun, Aug 22, 2010 at 5:36 PM, John_V <span dir="ltr"><<a href="mailto:jvillar.john@gmail.com">jvillar.john@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000"><div class="im">
Hi Lars,<br>
<br>
Thanks for the diagnosis and suggestion. Unfortunately it did not
work for me. <br>
<blockquote>getdp Lines -pre EleSta_v -cal -petsc_prealloc 50 -pos
Map<br>
</blockquote>
was only a little slower (52 s), but gave the same out-of-memory
error.<br>
<br>
I went as low as <br>
<blockquote>getdp Lines -pre EleSta_v -cal -petsc_prealloc 15 -pos
Map<br>
</blockquote>
This was very slow (~3 h), but the end point was the same.<br>
<br>
John<br>
<br>
On 8/20/2010 2:06 PM, Lars Rindorf wrote:
</div><div><div></div><div class="h5"><blockquote type="cite">
<div>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">Hi John</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<pre><span style="font-size: 11pt; color: rgb(31, 73, 125);" lang="EN-US">What goes wrong is that petsc tries to preallocate to much memory. Before your sparse matrix is constructet petsc ‘pre’allocates memory to speed up the assembly of the matrix. You can reduce the preallocated memory by the command line option:</span><span lang="EN-US"></span></pre>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "Courier New";">-petsc_prealloc
300</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "Courier New";" lang="EN-US">I
would try with 50. Assembly will probably be slow. Then I
would increase it.</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "Courier New";" lang="EN-US"> </span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "Courier New";" lang="EN-US">KR
Lars</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<div style="border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; padding: 3pt 0cm 0cm;">
<p class="MsoNormal"><b><span style="font-size: 10pt;">Fra:</span></b><span style="font-size: 10pt;"> John_V
[<a href="mailto:jvillar.john@gmail.com" target="_blank">mailto:jvillar.john@gmail.com</a>] <br>
<b>Sendt:</b> 20. august 2010 15:50<br>
<b>Til:</b> Lars Rindorf<br>
<b>Cc:</b> <a href="mailto:getdp@geuz.org" target="_blank">getdp@geuz.org</a><br>
<b>Emne:</b> Re: [Getdp] Out of memory error message</span></p>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Lars,<br>
<br>
The mesh file is large (148 MB), as you might expect since it
produces an
out-of-memory error. I'm not sure I should post something that
large to the
list, but you could regenerate it by 3-D mesh (using gmsh) of
the attached
Lines.geo file. That's how I did it. The remaining inputs are
also attached.<br>
<br>
Output follows:</p>
<div style="margin-left: 30pt;">
<p class="MsoNormal" style="margin-bottom: 12pt;">Info :
Running 'getdp Lines -pre EleSta_v -cal -pos Map'<br>
Info : Started on Fri Aug 20 09:29:56 2010<br>
Info : Loading problem definition 'Lines.pro'<br>
Info : Loading problem definition '<a href="http://parameters.pro" target="_blank">parameters.pro</a>'<br>
Info : Loading problem definition 'Jacobian_Lib.pro'<br>
Info : Loading problem definition 'Integration_Lib.pro'<br>
Info : Loading problem definition 'EleSta_v.pro'<br>
Info : Selected Resolution 'EleSta_v'<br>
Info : Loaded Geometric data 'Lines.msh'<br>
Info : System 'Sys_Ele' : Real<br>
P r e - P r o c e s s i n g . . .<br>
Info : Treatment Formulation 'Electrostatics_v'<br>
(0
%)
<br>
Info : Generate ExtendedGroup '_CO_Entity_13'
(NodesOf)<br>
(10
%)
<br>
(20
%)
<br>
(30
%)
<br>
(40
%)
<br>
(50
%)
<br>
(50
%)
<br>
(60
%)
<br>
(70
%)
<br>
(80
%)
<br>
(90
%)
<br>
(100
%)
<br>
Done!
<br>
Info : (CPU = 21.8594s)<br>
E n d P r e - P r o c e s s i n g<br>
P r o c e s s i n g . . .<br>
Info : Generate[Sys_Ele]<br>
(0
%)
<br>
(10
%)
<br>
(20
%)
<br>
(30
%)
<br>
(40
%)
<br>
(50
%)
<br>
(50 %)
<br>
(60
%)
<br>
(70
%)
<br>
(80
%)
<br>
(90
%)
<br>
(100
%)
<br>
Done!
<br>
Info : (CPU = 51.4063s)<br>
Info : Solve[Sys_Ele]<br>
Info : N: 390857<br>
<br>
[0]PETSC ERROR: --------------------- Error Message
------------------------------------<br>
[0]PETSC ERROR: Out of memory. This could be due to
allocating<br>
[0]PETSC ERROR: too large an object or bleeding by not
properly<br>
[0]PETSC ERROR: destroying unneeded objects.<br>
[0]PETSC ERROR: Memory allocated 0 Memory used by process 0<br>
[0]PETSC ERROR: Try running with -malloc_dump or -malloc_log
for info.<br>
[0]PETSC ERROR: Memory requested 943957912!<br>
[0]PETSC ERROR:
------------------------------------------------------------------------<br>
[0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 7, Mon
Jul 6 11:33:34
CDT 2009<br>
[0]PETSC ERROR: See docs/changes/index.html for recent
updates.<br>
[0]PETSC ERROR: See docs/faq.html for hints about trouble
shooting.<br>
[0]PETSC ERROR: See docs/index.html for manual pages.<br>
[0]PETSC ERROR:
------------------------------------------------------------------------<br>
[0]PETSC ERROR: Lines on a umfpack-c named GALILEO by
jvillar Fri Aug 20
09:29:56 2010<br>
[0]PETSC ERROR: Libraries linked from
/home/Administrator/src/petsc-3.0.0-p7/umfpack-cxx-opt/lib<br>
[0]PETSC ERROR: Configure run at Wed Mar 24 13:55:40 2010<br>
[0]PETSC ERROR: Configure options --with-debugging=0
--with-scalar-type=complex
--with-clanguage=cxx --with-shared=0 --with-mpi=0 --with-x=0
--with-fortran=0
--with-cc="gcc -mno-cygwin" --with-cxx="g++ -mno-cygwin"
COPTFLAGS=-O2 CXXOPTFLAGS=-O2 --with-umfpack=1
--download-umfpack=ifneeded
--useThreads=0 --useThreads=0<br>
[0]PETSC ERROR:
------------------------------------------------------------------------<br>
[0]PETSC ERROR: PetscMallocAlign() line 61 in
src/sys/memory/mal.c<br>
[0]PETSC ERROR: PetscFreeSpaceGet() line 14 in
src/mat/utils/freespace.c<br>
[0]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ() line 1322 in
src/mat/impls/aij/seq/aijfact.c<br>
[0]PETSC ERROR: MatILUFactorSymbolic() line 5308 in
src/mat/interface/matrix.c<br>
[0]PETSC ERROR: PCSetUp_ILU() line 293 in
src/ksp/pc/impls/factor/ilu/ilu.c<br>
[0]PETSC ERROR: PCSetUp() line 794 in
src/ksp/pc/interface/precon.c<br>
[0]PETSC ERROR: KSPSetUp() line 237 in
src/ksp/ksp/interface/itfunc.c<br>
[0]PETSC ERROR: KSPSolve() line 353 in
src/ksp/ksp/interface/itfunc.c<br>
[0]PETSC ERROR: User provided function() line 976 in
unknowndirectory/LinAlg_PETSC.cpp<br>
<br>
This application has requested the Runtime to terminate it
in an unusual way.<br>
Please contact the application's support team for more
information.</p>
</div>
<p class="MsoNormal" style="margin-bottom: 12pt;">John</p>
<div style="margin-left: 30pt;">
<p class="MsoNormal"> </p>
</div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">On Fri, Aug 20, 2010 at 1:57 AM, Lars
Rindorf <<a href="mailto:lrf@teknologisk.dk" target="_blank">lrf@teknologisk.dk</a>>
wrote:</p>
<p class="MsoNormal">Hi John<br>
<br>
Could you send the complete output and/or a simple file that
reproduces your
problem?<br>
<br>
KR Lars<br>
<br>
-----Oprindelig meddelelse-----<br>
Fra: <a href="mailto:getdp-bounces@ace20.montefiore.ulg.ac.be" target="_blank">getdp-bounces@ace20.montefiore.ulg.ac.be</a>
[mailto:<a href="mailto:getdp-bounces@ace20.montefiore.ulg.ac.be" target="_blank">getdp-bounces@ace20.montefiore.ulg.ac.be</a>]
På vegne af John_V<br>
Sendt: 19. august 2010 19:37<br>
Til: David Colignon<br>
Cc: <a href="mailto:getdp@geuz.org" target="_blank">getdp@geuz.org</a><br>
Emne: Re: [Getdp] Out of memory error message</p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom: 12pt;"><br>
Thank you Dave. I expected a possible barrier at 2 GB
and was<br>
surprised that the error message indicated only about
half that. I have<br>
also estimated memory usage by just watching the memory
used for that<br>
process in the Task Manager. This method's estimate is
consistent with<br>
the error message--it also indicates a bit over 1 GB
usage when the<br>
program fails. This is what made me wonder if the limit
were something<br>
else (maybe something more easily correctable). --John<br>
<br>
On 8/19/2010 3:00 AM, David Colignon wrote:<br>
> Hi John,<br>
><br>
> if you use the precompiled windows binary, I think
it is compiled in<br>
> 32 bits mode , and then you hit the 2 GB barrier.<br>
><br>
> Regards,<br>
><br>
> Dave<br>
></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">_______________________________________________<br>
getdp mailing list<br>
<a href="mailto:getdp@geuz.org" target="_blank">getdp@geuz.org</a><br>
<a href="http://www.geuz.org/mailman/listinfo/getdp" target="_blank">http://www.geuz.org/mailman/listinfo/getdp</a></p>
</div>
</div>
</div>
<p class="MsoNormal"> </p>
</div>
</blockquote>
</div></div></div>
</blockquote></div><br>