Opened 9 years ago

Last modified 5 years ago

#393 new feature

Fortran cross compilation requirement for manual cross files

Reported by: balaji Owned by:
Priority: minor Milestone: future
Component: mpich Keywords:
Cc: jratt@…, smithbr@…

Description (last modified by balaji)

This ticket is a reminder based on some input from Rob Latham about the use of cross files for cross compilation. Here's the summary of the problem:

Currently, the MPICH2 configuration files perform runtime checks to find type sizes and other parameters. During cross-compilation such runtime checks are not possible, so we expect the values to be provided to us in a cross file. However, it looks like there are some tricks to do this at compile time: http://trac.mcs.anl.gov/projects/parallel-netcdf/browser/trunk/configure.in#L207

We should follow a similar approach to remove or minimize the number of runtime checks we need, and thus not rely on having a user-provided cross file.

Attachments (1)

cross-compile.patch (22.9 KB) - added by balaji 8 years ago.

Download all attachments as: .zip

Change History (33)

comment:1 follow-up: Changed 9 years ago by balaji

  • Owner set to chan

comment:2 in reply to: ↑ 1 Changed 9 years ago by chan

  • Resolution set to wontfix
  • Status changed from new to closed

Replying to balaji:

autoconf's AC_CHECK_SIZEOF in cross-compile mode is already doing that.
Nothing to fix!

comment:3 Changed 9 years ago by balaji

  • Resolution wontfix deleted
  • Status changed from closed to reopened

The fact that a cross file needs to specify CROSS_SIZEOF_CHAR, etc., means that there are things we can fix. As I pointed out, probably everything cannot be fixed, but we can at least minimize the number of things required in the cross files. Here are some things which I think can be evaluated at compile time, and shouldn't be in cross files (picked from the BG/P cross compilation file):

CROSS_SIZEOF_CHAR=1
CROSS_SIZEOF_SHORT=2
CROSS_SIZEOF_INT=4
CROSS_SIZEOF_LONG=4
CROSS_SIZEOF_LONG_LONG=8
CROSS_SIZEOF_FLOAT=4
CROSS_SIZEOF_DOUBLE=8
CROSS_SIZEOF_LONG_DOUBLE=8
CROSS_SIZEOF_WCHAR_T=2
CROSS_SIZEOF_VOID_P=4
CROSS_SIZEOF_FLOAT_INT=8
CROSS_SIZEOF_DOUBLE_INT=16
CROSS_SIZEOF_LONG_INT=8
CROSS_SIZEOF_SHORT_INT=8
CROSS_SIZEOF_2_INT=8
CROSS_SIZEOF_LONG_DOUBLE_INT=16
CROSS_SIZEOF_LONG_LONG_INT=16
CROSS_SIZEOF_VOID_GREATER=no
CROSS_MPI_STATUS_SIZE=5

comment:4 follow-up: Changed 9 years ago by Anthony Chan


All the CROSS_SIZEOF_(1_type) are through AC_CHECK_SIZEOF,
they can be eliminated from cross file.
All the CROSS_SIZEOF_(2_TYPES) are troublesome, Bill does not use
AC_CHECK_SIZEOF, instead he writes a runtime test to check them,
don't think they can be eliminated.
CROSS_MPI_STATUS_SIZE is a Fortran to C constant, it requires a division.
has to be a runtime test in current form.  But I have been working to
get it as part of runtime initialization code

Besides, Functionally this part of code is working.  It will be
nice if we have the cycles to simplify things.   As I said in my previous
email on the cross-compile thread, the C data alignment tests
may not be cross-compile ready even on P.  If you want to spend
cycles on cross-compile support, I would say we should look at
those first.

Given the stuffs that I need to get done before/around April,
I don't think I will have any cycles left for this one any time soon.

A.Chan

----- "mpich2" <mpich2-maint@mcs.anl.gov> wrote:

> --------------------+-------------------------------------------------------
>  Reporter:  balaji  |        Owner:  chan
>      Type:  bug     |       Status:  reopened
>  Priority:  minor   |    Milestone:  mpich2-1.1b1
> Component:  mpich2  |   Resolution:
>  Keywords:          |
> --------------------+-------------------------------------------------------
>
> Changes (by balaji):
>
>   * status:  closed => reopened
>   * resolution:  wontfix =>
>
> Comment:
>
>  The fact that a cross file needs to specify CROSS_SIZEOF_CHAR, etc.,
> means
>  that there are things we can fix. As I pointed out, probably
> everything
>  cannot be fixed, but we can at least minimize the number of things
>  required in the cross files. Here are some things which I think can
> be
>  evaluated at compile time, and shouldn't be in cross files (picked
> from
>  the BG/P cross compilation file):
>
>  CROSS_SIZEOF_CHAR=1
>  CROSS_SIZEOF_SHORT=2
>  CROSS_SIZEOF_INT=4
>  CROSS_SIZEOF_LONG=4
>  CROSS_SIZEOF_LONG_LONG=8
>  CROSS_SIZEOF_FLOAT=4
>  CROSS_SIZEOF_DOUBLE=8
>  CROSS_SIZEOF_LONG_DOUBLE=8
>  CROSS_SIZEOF_WCHAR_T=2
>  CROSS_SIZEOF_VOID_P=4
>  CROSS_SIZEOF_FLOAT_INT=8
>  CROSS_SIZEOF_DOUBLE_INT=16
>  CROSS_SIZEOF_LONG_INT=8
>  CROSS_SIZEOF_SHORT_INT=8
>  CROSS_SIZEOF_2_INT=8
>  CROSS_SIZEOF_LONG_DOUBLE_INT=16
>  CROSS_SIZEOF_LONG_LONG_INT=16
>  CROSS_SIZEOF_VOID_GREATER=no
>  CROSS_MPI_STATUS_SIZE=5
>
> --
> Ticket URL:
> <https://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:3>

comment:5 in reply to: ↑ 4 Changed 9 years ago by goodell

Replying to [comment:4 Anthony Chan]:
>
> 
> All the CROSS_SIZEOF_(1_type) are through AC_CHECK_SIZEOF,
> they can be eliminated from cross file.
> All the CROSS_SIZEOF_(2_TYPES) are troublesome, Bill does not use
> AC_CHECK_SIZEOF, instead he writes a runtime test to check them,
> don't think they can be eliminated.
> CROSS_MPI_STATUS_SIZE is a Fortran to C constant, it requires a division.
> has to be a runtime test in current form.  But I have been working to
> get it as part of runtime initialization code

I'm not sure why you think that MPI_STATUS_SIZE needs to be a runtime test.
It's written that way right now but it could easily be rewritten to use the
pnetcdf trick instead.  For one thing you could evaluate the division in C
program at compile time and it would work fine, but that doesn't even matter.
The division is currently being handled in the configure script anyway, so
you could just keep it there.

There's also no good reason why the SIZEOF_(2_TYPES) checks can't be made
compile-time checks.  All that runtime test does is compute the result of:
sizeof(typeA)+sizeof(typeB).  We can do that easily enough at compile time.

> Besides, Functionally this part of code is working.  It will be
> nice if we have the cycles to simplify things.   As I said in my previous
> email on the cross-compile thread, the C data alignment tests
> may not be cross-compile ready even on P.  If you want to spend
> cycles on cross-compile support, I would say we should look at
> those first.

Actually, I'd get the easy and irritating stuff out of the way first.  The kind
of thing that irritates new users is when we don't let the computer do the
obvious and easy stuff like this.  If all they have to do is punch in the
alignment info when cross compiling then that's not too bad.  Having to specify
sizes for all the types just doesn't make very much sense.

> Given the stuffs that I need to get done before/around April,
> I don't think I will have any cycles left for this one any time soon.
> 
> A.Chan
>

comment:6 Changed 9 years ago by William Gropp

Agreed - the existing tests predated the quirky but effective compile-
time check.

Bill

On Jan 30, 2009, at 8:40 AM, mpich2 wrote:

> --------------------
> +-------------------------------------------------------
> Reporter:  balaji  |        Owner:  chan
>     Type:  bug     |       Status:  reopened
> Priority:  minor   |    Milestone:  mpich2-1.1b1
> Component:  mpich2  |   Resolution:
> Keywords:          |
> --------------------
> +-------------------------------------------------------
>
>
> Comment (by goodell):
>
> {{{
> Replying to [comment:4 Anthony Chan]:
>>
>>
>> All the CROSS_SIZEOF_(1_type) are through AC_CHECK_SIZEOF,
>> they can be eliminated from cross file.
>> All the CROSS_SIZEOF_(2_TYPES) are troublesome, Bill does not use
>> AC_CHECK_SIZEOF, instead he writes a runtime test to check them,
>> don't think they can be eliminated.
>> CROSS_MPI_STATUS_SIZE is a Fortran to C constant, it requires a
> division.
>> has to be a runtime test in current form.  But I have been working to
>> get it as part of runtime initialization code
>
> I'm not sure why you think that MPI_STATUS_SIZE needs to be a runtime
> test.
> It's written that way right now but it could easily be rewritten to
> use
> the
> pnetcdf trick instead.  For one thing you could evaluate the
> division in C
> program at compile time and it would work fine, but that doesn't even
> matter.
> The division is currently being handled in the configure script
> anyway, so
> you could just keep it there.
>
> There's also no good reason why the SIZEOF_(2_TYPES) checks can't be
> made
> compile-time checks.  All that runtime test does is compute the
> result of:
> sizeof(typeA)+sizeof(typeB).  We can do that easily enough at compile
> time.
>
>> Besides, Functionally this part of code is working.  It will be
>> nice if we have the cycles to simplify things.   As I said in my
> previous
>> email on the cross-compile thread, the C data alignment tests
>> may not be cross-compile ready even on P.  If you want to spend
>> cycles on cross-compile support, I would say we should look at
>> those first.
>
> Actually, I'd get the easy and irritating stuff out of the way
> first.  The
> kind
> of thing that irritates new users is when we don't let the computer
> do the
> obvious and easy stuff like this.  If all they have to do is punch
> in the
> alignment info when cross compiling then that's not too bad.  Having
> to
> specify
> sizes for all the types just doesn't make very much sense.
>
>> Given the stuffs that I need to get done before/around April,
>> I don't think I will have any cycles left for this one any time soon.
>>
>> A.Chan
>>
> }}}
>
> --
> Ticket URL: <https://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:5
> >

William Gropp
Deputy Director for Research
Institute for Advanced Computing Applications and Technologies
Paul and Cynthia Saylor Professor of Computer Science
University of Illinois Urbana-Champaign




comment:7 Changed 9 years ago by Anthony Chan


Hi Bill,

As Dave pointed out, the 2-type PAC test is essentially
sizeof(type1) + sizeof(type2).  Since we can get
sizeof(type1) from AC_CHECK_SIZEOF, does it mean that
we don't even need 2-type PAC test at all ?
I just want to make sure I am not missing something
important here...

A.Chan

----- "mpich2" <mpich2-maint@mcs.anl.gov> wrote:

> --------------------+-------------------------------------------------------
>  Reporter:  balaji  |        Owner:  chan
>      Type:  bug     |       Status:  reopened
>  Priority:  minor   |    Milestone:  mpich2-1.1b1
> Component:  mpich2  |   Resolution:
>  Keywords:          |
> --------------------+-------------------------------------------------------
>
>
> Comment (by William Gropp):
>
>  {{{
>
>  Agreed - the existing tests predated the quirky but effective
> compile-
>  time check.
>
>  Bill
>
>  On Jan 30, 2009, at 8:40 AM, mpich2 wrote:
>
>  > --------------------
>  > +-------------------------------------------------------
>  > Reporter:  balaji  |        Owner:  chan
>  >     Type:  bug     |       Status:  reopened
>  > Priority:  minor   |    Milestone:  mpich2-1.1b1
>  > Component:  mpich2  |   Resolution:
>  > Keywords:          |
>  > --------------------
>  > +-------------------------------------------------------
>  >
>  >
>  > Comment (by goodell):
>  >
>  > {{{
>  > Replying to [comment:4 Anthony Chan]:
>  >>
>  >>
>  >> All the CROSS_SIZEOF_(1_type) are through AC_CHECK_SIZEOF,
>  >> they can be eliminated from cross file.
>  >> All the CROSS_SIZEOF_(2_TYPES) are troublesome, Bill does not use
>  >> AC_CHECK_SIZEOF, instead he writes a runtime test to check them,
>  >> don't think they can be eliminated.
>  >> CROSS_MPI_STATUS_SIZE is a Fortran to C constant, it requires a
>  > division.
>  >> has to be a runtime test in current form.  But I have been working
> to
>  >> get it as part of runtime initialization code
>  >
>  > I'm not sure why you think that MPI_STATUS_SIZE needs to be a
> runtime
>  > test.
>  > It's written that way right now but it could easily be rewritten
> to
>  > use
>  > the
>  > pnetcdf trick instead.  For one thing you could evaluate the
>  > division in C
>  > program at compile time and it would work fine, but that doesn't
> even
>  > matter.
>  > The division is currently being handled in the configure script
>  > anyway, so
>  > you could just keep it there.
>  >
>  > There's also no good reason why the SIZEOF_(2_TYPES) checks can't
> be
>  > made
>  > compile-time checks.  All that runtime test does is compute the
>  > result of:
>  > sizeof(typeA)+sizeof(typeB).  We can do that easily enough at
> compile
>  > time.
>  >
>  >> Besides, Functionally this part of code is working.  It will be
>  >> nice if we have the cycles to simplify things.   As I said in my
>  > previous
>  >> email on the cross-compile thread, the C data alignment tests
>  >> may not be cross-compile ready even on P.  If you want to spend
>  >> cycles on cross-compile support, I would say we should look at
>  >> those first.
>  >
>  > Actually, I'd get the easy and irritating stuff out of the way
>  > first.  The
>  > kind
>  > of thing that irritates new users is when we don't let the
> computer
>  > do the
>  > obvious and easy stuff like this.  If all they have to do is punch
>  > in the
>  > alignment info when cross compiling then that's not too bad.
> Having
>  > to
>  > specify
>  > sizes for all the types just doesn't make very much sense.
>  >
>  >> Given the stuffs that I need to get done before/around April,
>  >> I don't think I will have any cycles left for this one any time
> soon.
>  >>
>  >> A.Chan
>  >>
>  > }}}
>  >
>  > --
>  > Ticket URL:
>  <https://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:5
>  > >
>
>  William Gropp
>  Deputy Director for Research
>  Institute for Advanced Computing Applications and Technologies
>  Paul and Cynthia Saylor Professor of Computer Science
>  University of Illinois Urbana-Champaign
>
>
>
>
>  }}}
>
> --
> Ticket URL:
> <https://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:>

comment:8 Changed 9 years ago by William Gropp

Yes, I think that the 2-type PAC routine is from long, long ago and
can be replaced with 2 AC_CHECK_SIZEOF and an eval (the short version
is to do that in aclocal_cc.m4).

Bill

On Feb 2, 2009, at 2:02 PM, mpich2 wrote:

> --------------------
> +-------------------------------------------------------
> Reporter:  balaji  |        Owner:  chan
>     Type:  bug     |       Status:  reopened
> Priority:  minor   |    Milestone:  mpich2-1.1b1
> Component:  mpich2  |   Resolution:
> Keywords:          |
> --------------------
> +-------------------------------------------------------
>
>
> Comment (by Anthony Chan):
>
> {{{
>
>
> Hi Bill,
>
> As Dave pointed out, the 2-type PAC test is essentially
> sizeof(type1) + sizeof(type2).  Since we can get
> sizeof(type1) from AC_CHECK_SIZEOF, does it mean that
> we don't even need 2-type PAC test at all ?
> I just want to make sure I am not missing something
> important here...
>
> A.Chan
>
> ----- "mpich2" <mpich2-maint@mcs.anl.gov> wrote:
>
>>
> --------------------
> +-------------------------------------------------------
>> Reporter:  balaji  |        Owner:  chan
>>     Type:  bug     |       Status:  reopened
>> Priority:  minor   |    Milestone:  mpich2-1.1b1
>> Component:  mpich2  |   Resolution:
>> Keywords:          |
>>
> --------------------
> +-------------------------------------------------------
>>
>>
>> Comment (by William Gropp):
>>
>> {{{
>>
>> Agreed - the existing tests predated the quirky but effective
>> compile-
>> time check.
>>
>> Bill
>>
>> On Jan 30, 2009, at 8:40 AM, mpich2 wrote:
>>
>>> --------------------
>>> +-------------------------------------------------------
>>> Reporter:  balaji  |        Owner:  chan
>>>    Type:  bug     |       Status:  reopened
>>> Priority:  minor   |    Milestone:  mpich2-1.1b1
>>> Component:  mpich2  |   Resolution:
>>> Keywords:          |
>>> --------------------
>>> +-------------------------------------------------------
>>>
>>>
>>> Comment (by goodell):
>>>
>>> {{{
>>> Replying to [comment:4 Anthony Chan]:
>>>>
>>>>
>>>> All the CROSS_SIZEOF_(1_type) are through AC_CHECK_SIZEOF,
>>>> they can be eliminated from cross file.
>>>> All the CROSS_SIZEOF_(2_TYPES) are troublesome, Bill does not use
>>>> AC_CHECK_SIZEOF, instead he writes a runtime test to check them,
>>>> don't think they can be eliminated.
>>>> CROSS_MPI_STATUS_SIZE is a Fortran to C constant, it requires a
>>> division.
>>>> has to be a runtime test in current form.  But I have been working
>> to
>>>> get it as part of runtime initialization code
>>>
>>> I'm not sure why you think that MPI_STATUS_SIZE needs to be a
>> runtime
>>> test.
>>> It's written that way right now but it could easily be rewritten
>> to
>>> use
>>> the
>>> pnetcdf trick instead.  For one thing you could evaluate the
>>> division in C
>>> program at compile time and it would work fine, but that doesn't
>> even
>>> matter.
>>> The division is currently being handled in the configure script
>>> anyway, so
>>> you could just keep it there.
>>>
>>> There's also no good reason why the SIZEOF_(2_TYPES) checks can't
>> be
>>> made
>>> compile-time checks.  All that runtime test does is compute the
>>> result of:
>>> sizeof(typeA)+sizeof(typeB).  We can do that easily enough at
>> compile
>>> time.
>>>
>>>> Besides, Functionally this part of code is working.  It will be
>>>> nice if we have the cycles to simplify things.   As I said in my
>>> previous
>>>> email on the cross-compile thread, the C data alignment tests
>>>> may not be cross-compile ready even on P.  If you want to spend
>>>> cycles on cross-compile support, I would say we should look at
>>>> those first.
>>>
>>> Actually, I'd get the easy and irritating stuff out of the way
>>> first.  The
>>> kind
>>> of thing that irritates new users is when we don't let the
>> computer
>>> do the
>>> obvious and easy stuff like this.  If all they have to do is punch
>>> in the
>>> alignment info when cross compiling then that's not too bad.
>> Having
>>> to
>>> specify
>>> sizes for all the types just doesn't make very much sense.
>>>
>>>> Given the stuffs that I need to get done before/around April,
>>>> I don't think I will have any cycles left for this one any time
>> soon.
>>>>
>>>> A.Chan
>>>>
>>> }}}
>>>
>>> --
>>> Ticket URL:
>> <https://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:5
>>>>
>>
>> William Gropp
>> Deputy Director for Research
>> Institute for Advanced Computing Applications and Technologies
>> Paul and Cynthia Saylor Professor of Computer Science
>> University of Illinois Urbana-Champaign
>>
>>
>>
>>
>> }}}
>>
>> --
>> Ticket URL:
>> <https://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:>
> }}}
>
> --
> Ticket URL: <https://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:
> >

William Gropp
Deputy Director for Research
Institute for Advanced Computing Applications and Technologies
Paul and Cynthia Saylor Professor of Computer Science
University of Illinois Urbana-Champaign




comment:9 Changed 9 years ago by Rob Ross

Hey,

Just to poke my nose in this to make sure I understand where things
are going, I see a runtime version of the pairtype size and extent
code in mpid_type_create_pairtype.c; I don't honestly know who wrote
it (probably me). Is this the version being discussed already?

I also wrote runtime versions of the alignment tests. These are not
currently used in MPICH2, but they are available in src/mpid/common/
datatype/dataloop/typesize_support.c. IIRC they are directly ported
from the configure tests; I planned to use them in other
instantiations of the dataloop code where I didn't want to have all
the configure tests.

Those versions might be useful in avoiding cross-compile issues, if
adopted across the board.

It has been quite a while since I messed with any of this, but I'm
happy to help if I can.

Please be careful with the alignment tests :)...

Rob

On Feb 2, 2009, at 4:34 PM, mpich2 wrote:

> --------------------
> +-------------------------------------------------------
> Reporter:  balaji  |        Owner:  chan
>     Type:  bug     |       Status:  reopened
> Priority:  minor   |    Milestone:  mpich2-1.1b1
> Component:  mpich2  |   Resolution:
> Keywords:          |
> --------------------
> +-------------------------------------------------------
>
>
> Comment (by William Gropp):
>
> {{{
>
> Yes, I think that the 2-type PAC routine is from long, long ago and
> can be replaced with 2 AC_CHECK_SIZEOF and an eval (the short version
> is to do that in aclocal_cc.m4).
>
> Bill
>
> On Feb 2, 2009, at 2:02 PM, mpich2 wrote:
>
>> --------------------
>> +-------------------------------------------------------
>> Reporter:  balaji  |        Owner:  chan
>>    Type:  bug     |       Status:  reopened
>> Priority:  minor   |    Milestone:  mpich2-1.1b1
>> Component:  mpich2  |   Resolution:
>> Keywords:          |
>> --------------------
>> +-------------------------------------------------------
>>
>>
>> Comment (by Anthony Chan):
>>
>> {{{
>>
>>
>> Hi Bill,
>>
>> As Dave pointed out, the 2-type PAC test is essentially
>> sizeof(type1) + sizeof(type2).  Since we can get
>> sizeof(type1) from AC_CHECK_SIZEOF, does it mean that
>> we don't even need 2-type PAC test at all ?
>> I just want to make sure I am not missing something
>> important here...
>>
>> A.Chan
>>
>> ----- "mpich2" <mpich2-maint@mcs.anl.gov> wrote:
>>
>>>
>> --------------------
>> +-------------------------------------------------------
>>> Reporter:  balaji  |        Owner:  chan
>>>    Type:  bug     |       Status:  reopened
>>> Priority:  minor   |    Milestone:  mpich2-1.1b1
>>> Component:  mpich2  |   Resolution:
>>> Keywords:          |
>>>
>> --------------------
>> +-------------------------------------------------------
>>>
>>>
>>> Comment (by William Gropp):
>>>
>>> {{{
>>>
>>> Agreed - the existing tests predated the quirky but effective
>>> compile-
>>> time check.
>>>
>>> Bill
>>>
>>> On Jan 30, 2009, at 8:40 AM, mpich2 wrote:
>>>
>>>> --------------------
>>>> +-------------------------------------------------------
>>>> Reporter:  balaji  |        Owner:  chan
>>>>   Type:  bug     |       Status:  reopened
>>>> Priority:  minor   |    Milestone:  mpich2-1.1b1
>>>> Component:  mpich2  |   Resolution:
>>>> Keywords:          |
>>>> --------------------
>>>> +-------------------------------------------------------
>>>>
>>>>
>>>> Comment (by goodell):
>>>>
>>>> {{{
>>>> Replying to [comment:4 Anthony Chan]:
>>>>>
>>>>>
>>>>> All the CROSS_SIZEOF_(1_type) are through AC_CHECK_SIZEOF,
>>>>> they can be eliminated from cross file.
>>>>> All the CROSS_SIZEOF_(2_TYPES) are troublesome, Bill does not use
>>>>> AC_CHECK_SIZEOF, instead he writes a runtime test to check them,
>>>>> don't think they can be eliminated.
>>>>> CROSS_MPI_STATUS_SIZE is a Fortran to C constant, it requires a
>>>> division.
>>>>> has to be a runtime test in current form.  But I have been working
>>> to
>>>>> get it as part of runtime initialization code
>>>>
>>>> I'm not sure why you think that MPI_STATUS_SIZE needs to be a
>>> runtime
>>>> test.
>>>> It's written that way right now but it could easily be rewritten
>>> to
>>>> use
>>>> the
>>>> pnetcdf trick instead.  For one thing you could evaluate the
>>>> division in C
>>>> program at compile time and it would work fine, but that doesn't
>>> even
>>>> matter.
>>>> The division is currently being handled in the configure script
>>>> anyway, so
>>>> you could just keep it there.
>>>>
>>>> There's also no good reason why the SIZEOF_(2_TYPES) checks can't
>>> be
>>>> made
>>>> compile-time checks.  All that runtime test does is compute the
>>>> result of:
>>>> sizeof(typeA)+sizeof(typeB).  We can do that easily enough at
>>> compile
>>>> time.
>>>>
>>>>> Besides, Functionally this part of code is working.  It will be
>>>>> nice if we have the cycles to simplify things.   As I said in my
>>>> previous
>>>>> email on the cross-compile thread, the C data alignment tests
>>>>> may not be cross-compile ready even on P.  If you want to spend
>>>>> cycles on cross-compile support, I would say we should look at
>>>>> those first.
>>>>
>>>> Actually, I'd get the easy and irritating stuff out of the way
>>>> first.  The
>>>> kind
>>>> of thing that irritates new users is when we don't let the
>>> computer
>>>> do the
>>>> obvious and easy stuff like this.  If all they have to do is punch
>>>> in the
>>>> alignment info when cross compiling then that's not too bad.
>>> Having
>>>> to
>>>> specify
>>>> sizes for all the types just doesn't make very much sense.
>>>>
>>>>> Given the stuffs that I need to get done before/around April,
>>>>> I don't think I will have any cycles left for this one any time
>>> soon.
>>>>>
>>>>> A.Chan
>>>>>

comment:10 Changed 9 years ago by Anthony Chan

Hi Rob,

> I see a runtime version of the pairtype size and extent
> code in mpid_type_create_pairtype.c; I don't honestly know who wrote
> it (probably me). Is this the version being discussed already?

No, I don't think our discussion has ever gone that deep.
What does mpid_type_create_pairtype.c need from configure ?
I see one thing that configure may not be able to handle is

 struct { ut1_ a; ut2_ b; } foo;
 ....
 true_ub = (MPI_VOID_PTR_CAST_TO_MPI_AINT ((char *) &foo.b -     \
                                           (char *) &foo.a)) +   \
           (MPI_Aint) sizeof(foo.b);

>  I also wrote runtime versions of the alignment tests. These are not
>  currently used in MPICH2, but they are available in src/mpid/common/
>  datatype/dataloop/typesize_support.c. IIRC they are directly ported
>  from the configure tests; I planned to use them in other
>  instantiations of the dataloop code where I didn't want to have all
>  the configure tests.

Do you think typesize_support.c can be easily integrated into
maint/getcross.c's main() that user can run on the compute node
to create a cross file ?

A.Chan

comment:11 Changed 9 years ago by goodell

Anthony,

Based on discussions at lunch today, I'm pretty sure that Rob is suggesting that we change the code to no longer require configure-time alignment checks. Instead we will replace them with a runtime check at startup. This might depend on whether or not we are using the alignment information outside of the dataloop code.

If that doesn't make sense, I can try to clarify.

-Dave

comment:12 Changed 9 years ago by Anthony Chan

Hi Dave,

That makes perfect sense, I assume this includes the
MPI 2-type ?  I am working on moving all MPE configure
runtime tests into MPE's startup code as well
so that MPE configure will almost be free of runtime tests
(i.e. except CROSS_BIGENDIAN).

Wonder if the same can be done on the CROSS_F77_*
and CROSS_F90_* in MPICH2 ?

A.Chan
----- "mpich2" <mpich2-maint@mcs.anl.gov> wrote:

> --------------------+-------------------------------------------------------
>  Reporter:  balaji  |        Owner:  chan
>      Type:  bug     |       Status:  reopened
>  Priority:  minor   |    Milestone:  mpich2-1.1b1
> Component:  mpich2  |   Resolution:
>  Keywords:          |
> --------------------+-------------------------------------------------------
>
>
> Comment (by goodell):
>
>  Anthony,
>
>  Based on discussions at lunch today, I'm pretty sure that Rob is
>  suggesting that we change the code to no longer require
> configure-time
>  alignment checks.  Instead we will replace them with a runtime check
> at
>  startup.  This might depend on whether or not we are using the
> alignment
>  information outside of the dataloop code.
>
>  If that doesn't make sense, I can try to clarify.
>
>  -Dave
>
> --
> Ticket URL:
> <https://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:11>

comment:13 Changed 9 years ago by balaji

  • Milestone mpich2-1.1b1 deleted

comment:14 Changed 9 years ago by balaji

  • Milestone set to mpich2-1.1.1

comment:15 Changed 8 years ago by chan

  • Milestone changed from mpich2-1.1.1 to future

comment:16 Changed 8 years ago by balaji

Since we are using newer versions of autoconf, AC_CHECK_SIZEOF works correctly (with compile-time only checks) for inbuilt as well as structure datatypes. I have already moved the Bsend overhead structure to use this. I think we are ready to get rid of some of the cross-compile variables:

CROSS_SIZEOF_CHAR=1
CROSS_SIZEOF_SHORT=2
CROSS_SIZEOF_INT=4
CROSS_SIZEOF_LONG=4
CROSS_SIZEOF_LONG_LONG=8
CROSS_SIZEOF_FLOAT=4
CROSS_SIZEOF_DOUBLE=8
CROSS_SIZEOF_LONG_DOUBLE=8
CROSS_SIZEOF_WCHAR_T=2
CROSS_SIZEOF_VOID_P=4
CROSS_SIZEOF_FLOAT_INT=8
CROSS_SIZEOF_DOUBLE_INT=16
CROSS_SIZEOF_LONG_INT=8
CROSS_SIZEOF_SHORT_INT=8
CROSS_SIZEOF_2_INT=8
CROSS_SIZEOF_LONG_DOUBLE_INT=16
CROSS_SIZEOF_LONG_LONG_INT=16

... and maybe more too.

comment:17 Changed 8 years ago by balaji

  • Cc jratt@…, smithbr@… added
  • Milestone changed from future to mpich2-1.1.2
  • Owner changed from chan to balaji
  • Status changed from reopened to assigned

Moving this up to 1.1.2, since it was reported as a problem by IBM. It'll be good to have this fixed and possibly have no-cross file usage at some point.

comment:18 Changed 8 years ago by balaji

Got rid of a bunch of cross-compile variables in [6ecaceeb489ab6201ef7254209539c7bd0cd8354]. Some more Fortran-related cross-compile stuff is still pending, and the alignment rules are not cross-compile safe yet either.

comment:19 Changed 8 years ago by balaji

  • Owner changed from balaji to chan
  • Summary changed from Cross compilation requirement for manual cross files to Fortran cross compilation requirement for manual cross files

As of [5eb5a689cf097e29581aa7b63a51267b03171ea2], we should be fully cross-compile safe for C without requiring any cross file definitions. Fortran still requires some variables to be defined in the cross file.

Handing over the Fortran stuff to Anthony as I'm a Fortran illiterate.

comment:20 Changed 8 years ago by chan

Since all the CROSS_SIZEOF_* are removed from AC_CHECK_SIZEOF, the AC_PREREQs need to be updated (Most still depends on 2.59). Most mpich2 configure.ins ain't standalone, (maint/updatefiles enforces autoconf version already), should we just remove the AC_PREREQ in non-standalone configure.in ?

comment:21 Changed 8 years ago by balaji

Agreed. AC_PREREQs can be pulled out for non-standalone configures. Only MPE, ROMIO and Hydra need to have their own AC_PREREQs.

comment:22 Changed 8 years ago by balaji

Now that the regular sizeof type CROSS variables have been gotten rid of, I was looking into the alignment macros. I see a note above from RobR about these, so I want to make sure I'm not missing something about them.

Most of these tests are AC_TRY_RUN tests, which is painful for cross-compilation. But looking into the runtime tests themselves, they seem to be just a bunch of sizeof comparisons. Is there a reason this needs to be done as a AC_TRY_RUN test instead of a true compile-time check? I've added a macro (PAC_STRUCT_ALIGNMENT: http://trac.mcs.anl.gov/projects/mpich2/browser/mpich2/trunk/confdb/aclocal_cc.m4?rev=5177#L1456) that gives an example of how this can be done at compile-time only. If no one has any issues with this, I'd like to change all these macros to something similar.

Also, there is a lot of repetition in all these macros. Can we merge them into a single macro and get them to #define appropriate values as needed? It's much easier to maintain and will minimize bugs.

comment:23 Changed 8 years ago by goodell

Based on prior discussion with RobR I don't think that these can be made compile-time only checks. I'd have to think about it for a little while to be 100% certain about this but RobR seemed pretty sure at the time. Rob's suggestion was to convert them to runtime checks in the library that occur at initialization time.

It might be best to hold off a little while on this until you and I can jointly talk with RobR face-to-face and (hopefully) settle this issue for good. If we are all on the same page we can fix the problem correctly and comment the heck out of it so that we can remember why this subtle yet important issue should be handled the way it is.

comment:24 Changed 8 years ago by rross@…

There's example code for doing type size checking in:
   mpich2/src/mpid/common/datatype/dataloop/typesize_support.c

I think this is related?

Rob

On Aug 10, 2009, at 1:30 PM, mpich2 wrote:

> #393: Fortran cross compilation requirement for manual cross files
> --------------------
> +-------------------------------------------------------
> Reporter:  balaji  |        Owner:  chan
>     Type:  bug     |       Status:  assigned
> Priority:  minor   |    Milestone:  mpich2-1.1.2
> Component:  mpich2  |   Resolution:
> Keywords:          |
> --------------------
> +-------------------------------------------------------
>
> Comment(by goodell):
>
> Based on prior discussion with RobR I don't think that these can be
> made
> compile-time only checks.  I'd have to think about it for a little
> while
> to be 100% certain about this but RobR seemed pretty sure at the time.
> Rob's suggestion was to convert them to runtime checks in the
> library that
> occur at initialization time.
>
> It might be best to hold off a little while on this until you and I
> can
> jointly talk with RobR face-to-face and (hopefully) settle this
> issue for
> good.  If we are all on the same page we can fix the problem
> correctly and
> comment the heck out of it so that we can remember why this subtle yet
> important issue should be handled the way it is.
>
> --
> Ticket URL: <http://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:23
> >
> mpich2 <http://www.mcs.anl.gov/research/projects/mpich2>
> MPICH2 is a high performance and widely portable implementation of
> the Message Passing Interface (MPI) standard.


Changed 8 years ago by balaji

comment:25 Changed 8 years ago by balaji

Yeah, I did look at the code in mpich2/src/mpid/common/datatype/dataloop/typesize_support.c for this, and I do believe these can be made compile-time checks. I've attached a patch for some of the macros. This is not complete, but can serve as a starting point.

comment:26 Changed 8 years ago by rross@…

Hi,

Those used to be configure-time checks. I'm not sure how you make them
compile-time, because many of them require instantiating structures in
order to look at sizes, etc?

Rob

On Aug 10, 2009, at 1:56 PM, mpich2 wrote:

> #393: Fortran cross compilation requirement for manual cross files
> --------------------
> +-------------------------------------------------------
> Reporter:  balaji  |        Owner:  chan
>     Type:  bug     |       Status:  assigned
> Priority:  minor   |    Milestone:  mpich2-1.1.2
> Component:  mpich2  |   Resolution:
> Keywords:          |
> --------------------
> +-------------------------------------------------------
>
> Comment(by balaji):
>
> Yeah, I did look at the code in
> mpich2/src/mpid/common/datatype/dataloop/typesize_support.c for
> this, and
> I do believe these can be made compile-time checks. I've attached a
> patch
> for some of the macros. This is not complete, but can serve as a
> starting
> point.
>
> --
> Ticket URL: <http://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:25
> >
> mpich2 <http://www.mcs.anl.gov/research/projects/mpich2>
> MPICH2 is a high performance and widely portable implementation of
> the Message Passing Interface (MPI) standard.


comment:27 Changed 8 years ago by balaji

The new autoconf versions allow you to check sizeof at compile-time for structures as well. They do a check similar to http://trac.mcs.anl.gov/projects/parallel-netcdf/browser/trunk/configure.in#L207 but do a binary search for sizes of structures.

comment:28 Changed 8 years ago by rross@…

Ok, I'll take your word for it. Run-time is pretty straight-forward
though. -- Rob

On Aug 10, 2009, at 2:00 PM, mpich2 wrote:

> #393: Fortran cross compilation requirement for manual cross files
> --------------------
> +-------------------------------------------------------
> Reporter:  balaji  |        Owner:  chan
>     Type:  bug     |       Status:  assigned
> Priority:  minor   |    Milestone:  mpich2-1.1.2
> Component:  mpich2  |   Resolution:
> Keywords:          |
> --------------------
> +-------------------------------------------------------
>
> Comment(by balaji):
>
> The new autoconf versions allow you to check sizeof at compile-time
> for
> structures as well. They do a check similar to
> http://trac.mcs.anl.gov/projects/parallel-
> netcdf/browser/trunk/configure.in#L207 but do a binary search for
> sizes of
> structures.
>
> --
> Ticket URL: <http://trac.mcs.anl.gov/projects/mpich2/ticket/393#comment:27
> >
> mpich2 <http://www.mcs.anl.gov/research/projects/mpich2>
> MPICH2 is a high performance and widely portable implementation of
> the Message Passing Interface (MPI) standard.


comment:29 Changed 8 years ago by balaji

We are already doing this for bsend overhead in the top-level configure; I don't see anything fundamentally wrong with the way autoconf is doing this. Runtime tests might be OK too, but compile-time might be preferable if we don't want to add more init-time overhead (though it might not be too much). Either approach is fine with me, as long as we can get rid of the AC_TRY_RUNs and the need to maintain large cross-compile files.

comment:30 Changed 8 years ago by balaji

  • Milestone changed from mpich2-1.1.2 to mpich2-1.2

Milestone mpich2-1.1.2 deleted

comment:31 Changed 8 years ago by chan

  • Milestone changed from mpich2-1.2.1 to future
  • Owner changed from chan to balaji
  • Type changed from bug to feature

Assign it to Pavan for the C aligment stuffs

comment:32 Changed 5 years ago by balaji

  • Description modified (diff)
  • Owner balaji deleted
  • Status changed from assigned to new
Note: See TracTickets for help on using tickets.