Implementation experience with SIA 2

[François Bonnarel]

Hi Walter,
Hi all,
    I will integrate your points on the RFC page. that's the way to 
express concerns at this step of the process
    Some of your points got answers in July. I will also integrate them.
    We will see what must still be considered.
    A new verson is in preparation anyway.
     It will hopefully be posted before Interop,and you can check this 
one about your concerns
    I encourage you to present your experience (even with demos) at the 
appropriate sessions in banff if you are coming
    I don't think we will be ready for the TCG review  before Interop anyway
Best regards
François
Le 27/09/2014 01:28, Walter Landry a écrit :
> Hello Everyone,
>
> Back in July, I sent a note to this list about some issues I had with
> SIA 2.0.  Since then, we have implemented a synthetic image generation
> service for the Planck satellite.  We tried to implement this is in a
> way consistent with SIA 2.0, but we had some difficulties.
>
> 1) BAND
>
>     The Planck satellite detector bands are all specified in GHz: 30,
>     44, 70, etc.  These are nice, integer numbers.  Mapping to
>     wavelength leaves me with numbers that are not exactly
>     representable in floating point.  This means that every search has
>     to give a range.  It would be nice if I could specify the frequency
>     instead of the wavelength.  We ended up using the keyword FREQ in
>     MHz, since I do not know of any astronomical observations of EM
>     radiation that go lower than that.
>
> 2) RANGE
>
>     As I mentioned in July, RANGE is prohibitively expensive for this
>     data set.  So we do not support it, will never support it, and I
>     still think it should not be part of the spec.
>
> 3) POS
>
>     Since this is a synthetic image generation service, it would be
>     nice to make rectangular images.  The current SIA 2.0 spec has no
>     great circle rectangles.  The client has to construct the polygons
>     themselves, which is non-trivial.  Given the negative reaction I
>     got to Box's last time, we ended up ditching SIA 2.0 for this
>     entirely and using SIA 1 syntax: POS, SIZE, CFRAME, CDELT (though
>     SPATRES would have been fine).  I would really prefer a better
>     mechanism than this.
>
> 4) TARGET vs OBJECT
>
>     Why does SIA 2.0 use the TARGET parameter?  OBJECT is an existing
>     standard FITS convention.
>
> 5) Syntax
>
>     Consider these issues:
>
>     a) In July, I highlighted a problem with the syntax of POS
>        parameters.  It requires spaces, which must be URL encoded or
>        things silently break.  Silent breakage is the worst kind of
>        breakage.
>
>     b) Polygon searches use a straight list of numbers.  It would be
>        better to have a list of pairs to make typos more obvious.
>
>     c) We need to be able to select multiple detectors at once, so we
>        would like to have an array of strings.
>
>     d) There is no way to add arbitrary parameters.  COORD was the way
>        to do that in old versions of SIA 2.0.  Now I have to use up a
>        keyword and hope it does not accidentally conflict with new
>        versions of the standard.  This is not going to scale.
>
>     e) We have a smart client doing searches on behalf of the user.  In
>        general, we would like to set arbitrary metadata that are not
>        necessary for the search but convenient for the user.
>
>     This prompted me to use a more general syntax to express queries.
>     Specifically, I used json5
>
>       https://github.com/aseemk/json5
>
>     It is an extension of JSON to make it friendlier to write.  It is a
>     strict superset of JSON and a strict subset of Javascript.  So
>     every valid JSON file is valid json5, and eval() will still work
>     for those of you foolish enough to run it on unverified user input ;)
>     To be specific, a sample query would be
>
>       http://irsa.ipac.caltech.edu/cgi-bin/Planck_TOI/nph-planck_toi_sia?POS=[0.053,-0.062]&CFRAME='GAL'&ROTANG=90&SIZE=1&CDELT=0.05&FREQ=44000&ITERATIONS=20&INSTRUMENT=['24m','24s']&TIME=[[0,55300],[55500,Infinity]]&USER_METADATA={CLIENT:'IRSA Smart Client'}
>
>     Note that the service is not public yet, so this URL will not work
>     for you yet.
>
>     Internally, every parameter is converted into a json5 element.  So
>     this would turn into the json5 document
>
>     {
>       POS:[0.053,-0.062],
>       CFRAME:'GAL',
>       ROTANG:90,
>       SIZE:1,
>       CDELT:0.05,
>       FREQ:44000,
>       ITERATIONS:20,
>       INSTRUMENT:['24m','24s'],
>       TIME:[[0,55300],[55500,Infinity]],
>       USER_METADATA:{CLIENT:'IRSA Smart Client'}
>     }
>
>     Modulo whitespace, this is just replacing '&' with ',' and '=' with
>     ':'.  We also support submitting a json5 document directly
>
>       http://irsa.ipac.caltech.edu/cgi-bin/Planck_TOI/nph-planck_toi_sia?{POS:[0.053,-0.062],CFRAME:'GAL',ROTANG:90,SIZE:1,CDELT:0.05,FREQ:44000,ITERATIONS:20,INSTRUMENT:['24m','24s'],TIME:[[0,55300],[55500,Infinity]],USER_METADATA:{CLIENT:'IRSA Smart Client'}}
>
>     On a side note, I have used JSON (not json5) as input for
>     simulations in geology [1].  The SAMRAI Adaptive Mesh Refinement
>     framework for massively parallel simulations [2] also uses a format
>     that is almost indistinguishable [3] from json5 for input files.
>     So I would claim that json5 would be able to cover any needs that
>     SIMDAL would need in specifying model parameters.
>
> Given all of the issues that I ran into, it is not clear to me that it
> would be a good idea to ratify SIA 2 as it is now.  Whether or not you
> like the changes I made, there seem to be some major deficiencies that
> need to be addressed before the standard can be accepted.
>
> Cheers,
> Walter Landry
>
>
> [1] http://geodynamics.org/cig/software/gale
> [2] https://computation-rnd.llnl.gov/SAMRAI/index.php
> [3] You can separate elements in an array or object with newlines instead of commas.