changes/updates in regards to spr based error estimation/adaptive solve#41
Conversation
Also makes some of the print infos more readable
There are a couple of new member functions to this class to identify if a field is marked for solution transfer or not.
More clean up and testing needed
transfer The fields that will be added to solution transfer will be specified on the fortran side
To make them accessible on the fortran side
The optional input prefix (of type character string) can be used to name the field on the C++ side. This commit also adds fortran APIs to mark fields that need to be transfered.
We will need to figure out a better way of categorizing all this different cases, at some point
This is used to mark fields for solution transfer using the same logic they are written to hdf5 restart files
and also mark them for solution transfer The vectors defined in this file seem to be all needed in each solution step so they must all be transferred during mesh adapt
to give all those fields proper names. this is done only for a 2d example and more may need to be done for other examples. since the new "prefix" argument in create_field api is optional this should not break the other tests cases.
844e5f3 to
99aead8
Compare
to be consistent with the latest changes on SCOREC/core
Conflicts: m3dc1_scorec/api/m3dc1_scorec.cc m3dc1_scorec/src/m3dc1_adapt.cc unstructured/newpar.f90
Note that something does not seem to work in parallel. Probably some data structure changes that is needed for m3dc1 to work that your are missing when doing spr-adapt
For the case of 3D these are handled outside of this loop according to which fields have to be transfered during solution transfer
This is called after remove3D to initialize the data-structures on the empty parts of the mesh, to avoid problems with entity orders when restore3D is called.
|
3D adapt development using spr for error estimation are also added to this pull-request. The main API is the same and works for both 2D and 3D. |
|
@mortezah Excellent work. If you think it's ready, I can try using it myself. I tried compiling on stellar though and it doesn't look like the version of the SCOREC routines has been appropriately updated. Presumably they need to be installed and then linked to in If you want to try running a case, here's a 2D example on stellar based on a benchmark we're doing with NIMROD: |
|
@bclyons12. Yes, you are right, since I have made changes to the C++ side of the code as well as to the Fortran side, one needs to build |
|
@bclyons12, please check out the latest M3DC1 and compile it with SCORECVER=adapt using stellart.mk. I won't update the existing SCOREC libraries until everyone agrees to do so. So 3D adaptation will be available only with SCORECVER=adapt for a while. Also, please let me know if you have any other machines to test out 3D adaptation. |
I think this is going to cause problems. By merging this branch into master all the latest changes in the Fortran side are there and if the m3dc1_scorec installation is not updated to use the latest changes, the build of m3dc1 executables is going to fail on every machine except stellar. Also, we should avoid introducing extra flags such as SCORECVER etc, which makes keeping track of things difficult. All the changes that I have made in this branch are consistent and if they are all used properly everything should work and there should be no need for the additional flag in the build process. |
|
Given that, are you saying it should be updated everywhere? If yes, are we pretty sure that will not be an issue? We should get this resolved quickly. |
|
@MarkShephard yes that is true, m3dc1_scorec (which is the C++ part of M3DC1) has to be updated everywhere. I introduced new APIs there that are called from the Fortran side, and if the Fortran side is compiled against the old version of m3dc1_scorec the compilation will fail. |
|
@mortezah @MarkShephard @seegyoung If this passes all regression tests, then I'm fine with the merge, but yes, we need to get the installation updated on all machines |
|
@bclyons12 @MarkShephard @seegyoung since I do not have access to all the machines I cannot confirm if the regression tests pass on all the machines. That was the main motivation behind having this reviewed before being merged. |
|
Having a unified build system would be very helpful here. As it is now, I don’t think anyone working on the Fortran side of things knows how to compile the SCOREC libraries to be able to test them.
… On Apr 14, 2022, at 1:04 PM, Morteza ***@***.***> wrote:
@bclyons12 <https://github.com/bclyons12> @MarkShephard <https://github.com/MarkShephard> @seegyoung <https://github.com/seegyoung> since I do not have access to all the machines I can confirm if the regression tests pass on all the machines. That was the main motivation behind having this reviewed before being merged.
—
Reply to this email directly, view it on GitHub <#41 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAE2OL4LZARMOZR6SOFRYCLVFBFZTANCNFSM5EL4C65Q>.
You are receiving this because your review was requested.
|
|
@nferraro @bclyons12 @MarkShephard @seegyoung @sjardin I had a feeling that treating |
|
This might be a longer discussion that needs input from others. In the
mean time, I would really appreciate it if the version in git master
compiled, as I am trying to make some other changes and cannot compile as
of now.
…On Thu, Apr 14, 2022 at 2:03 PM Morteza ***@***.***> wrote:
@nferraro <https://github.com/nferraro> @bclyons12
<https://github.com/bclyons12> @MarkShephard
<https://github.com/MarkShephard> @seegyoung
<https://github.com/seegyoung> @sjardin <https://github.com/sjardin>
I had a feeling that treating m3dc1_scorec as an external library is not
ideal (since every time that is changed installations have to update
everywhere). I was actually preparing a document (
https://docs.google.com/document/d/10TkOtAaXKsX6s6reRhewc2rFYAw0y9EGMT1CkPcfJ7A/edit?usp=sharing)
to suggest doing things differently which I guess is also consistent with
what you call unified build systems. It would be good if you can take a
look at that document and let me know if that is what you had in mind. Once
that is agreed on, I can either update the current build system (using
make) or move entirely to cmake to not only build the final Fortran
executables but also build a local version of the m3dc1_scorec library to
link against.
—
Reply to this email directly, view it on GitHub
<#41 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIIVKSBSQT5Z5QOGUDRRVVLVFBMUNANCNFSM5EL4C65Q>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
609 243 2635
|
|
@sjardin @bclyons12 @nferraro @MarkShephard I will revert the merge, as I don't know how soon Seegyoung can install the updated libraries. Once we have a more streamlined way of m3dc1_scorec libraries I can then merge this branch back into master |
@sjardin I have just reverted the merge. If you do a git pull you should be able to compile. Let me know if there are any issues. |
|
My apologies for causing any trouble. I wasn't aware that Morteza's branch is not ready for the merge. Brendan can install Morteza' branch (Fortran code) only with SCORECVER=adapt. And I assume I don't have to do any thing special for now (Morteza, please correct me if I am wrong). |
|
As @nferraro, I think is a good time to target a unified build between |
|
Yes, treating the scorec libraries as an internal dependency rather than an external library makes sense to me. Ideally, we should just be able to type “make all” in the git directory and everything would build together.
I also agree with Steve that we should try to make sure that the master branch always builds and passes the regression tests using the default build configuration.
… On Apr 14, 2022, at 3:49 PM, sjardin ***@***.***> wrote:
This might be a longer discussion that needs input from others. In the
mean time, I would really appreciate it if the version in git master
compiled, as I am trying to make some other changes and cannot compile as
of now.
On Thu, Apr 14, 2022 at 2:03 PM Morteza ***@***.***> wrote:
> @nferraro <https://github.com/nferraro> @bclyons12
> <https://github.com/bclyons12> @MarkShephard
> <https://github.com/MarkShephard> @seegyoung
> <https://github.com/seegyoung> @sjardin <https://github.com/sjardin>
>
> I had a feeling that treating m3dc1_scorec as an external library is not
> ideal (since every time that is changed installations have to update
> everywhere). I was actually preparing a document (
> https://docs.google.com/document/d/10TkOtAaXKsX6s6reRhewc2rFYAw0y9EGMT1CkPcfJ7A/edit?usp=sharing)
> to suggest doing things differently which I guess is also consistent with
> what you call unified build systems. It would be good if you can take a
> look at that document and let me know if that is what you had in mind. Once
> that is agreed on, I can either update the current build system (using
> make) or move entirely to cmake to not only build the final Fortran
> executables but also build a local version of the m3dc1_scorec library to
> link against.
>
> —
> Reply to this email directly, view it on GitHub
> <#41 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AIIVKSBSQT5Z5QOGUDRRVVLVFBMUNANCNFSM5EL4C65Q>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
609 243 2635
—
Reply to this email directly, view it on GitHub <#41 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAE2OL4FBWTDUP42AMYO62TVFBZE3ANCNFSM5EL4C65Q>.
You are receiving this because you were mentioned.
|
I see two possible ways to achieve this:
The second approach is the cleaner approach but requires a good deal of time/effort to put in place. |
|
Unified build is working on stellar with #46. Need to review these changes then update accordingly for all systems. |
6 participants
Summary
The changes in this PR are in regards to adaptive-solve implementation using spr-based error estimation. Additional updates/modifications are also included for things like solution transfer/field transfer/etc (see the detailed list at the end of this notes)
At this point, things work for both 2D/3D. There are a few new parameters introduced in the input file as follows:
ispradapt(default 0): 1 to turn on spr-adapt and 0 to turn it offisprntime(default 10): an integer bigger than 0 to call spr-adapt everyisprntimetime stepsisprweight(default 0.1): a real number smaller than 1 specifying the weight of error to calculate the size-field (note smaller weight results in much more aggressive adaptation).isprmaxsize(default 0.05): user-specified maximum allowed edge length in the entire meshisprrefinelevel(default 1): maximum level of refinement allowed during each call to spr-adapt [for example a value of 3 means the lengths of the edges will not reduce by a factor of more than 8(= 2^3) in each call to spr-adaptisprcoarsenlevel(default -1): maximum level of coarsening allowed during each call to spr-adapt. use -1 to disable coarsening altogether [for example a value of 3 means the lengths of the edges will not increase by a factor of more than 8(= 2^3) in each call to spr-adaptThe high-level API that can be called on the Fortran side is
adapt_by_spr. An example use is as followswhere
field_veccontains the list of all the privacy variables (e.g.u_field,vz_field,chi_field,psi_field, etc) and the the second argumentpsi_gtellsadapt_by_sprto use thepsi_fieldfor error estimation.Checklist
create_fieldandcreate_vectorsubroutines to take an optionalprefixargument as the field nameoutput.f90to mark the fields for solution transfer following the same logic used to write fields to restart