A note on a possible extension to SIAP
Pedro Osuna
posuna at iso.vilspa.esa.es
Tue Apr 20 06:16:46 PDT 2004
Dear Francois,
this new approach, where you include different tables in the same
RESOURCE results, is closer to what we propose.
Although we like it more than your previous proposal, we have been
looking at the pros and cons of your approach vs ours, and these are the
conclusions we have drawn. We think there is no-single solution for this
issue, but also believe it is good to post these ideas to the whole
community for discussion. Here they go:
In your example, a table "points" to a structure which is outside itself
(another table). This is to avoid modification of the DTD to allow
tables inside tables. All the tables appear, therefore, at the same
level.
This means that the external tables do have to have an identifier to be
able to do the joins among tables.
This would be an exact mapping of how an RDBMS works: independent tables
joined through identifiers.
The problem with this approach is that it leaves the whole of the
interpretation of the structure to the client.
As the service is hiding the structure (it does not have any structure
by itself, only pointers and identifiers to be able to do joins), the
client will have to re-interpret it. In the case of a couple of
observations, this could be done fast. In the case of hundreds of
observations, in your approach the client would have to go through the
whole results for each and every observation to figure out which
exposures belong to it (and later, for each and every exposure to find
the sources, etc.).
As the results given in your way do not contain any specific ordering,
the display client will have to make this ordering by itself. This
reordering would be done _without_ indexes (in the database terminology
sense) and therefore would be a very un-efficient task. In our case,
however, indexes are implicit in the structure itself, as
"substructures" (as exposures, sources, etc.) are already inserted below
a specific "superstructure" (like observation) and therefore the client
only travels the structures once.
In our proposal, therefore, the join issues are handled at the server
level, and the structure is already unveiled to the client, which would
only have to display it in one go.
Another "artifact" of your proposal is the use of those "dummy"
placeholders:
<TD>Exposure_Table</TD>
These ones appear due to the fact that VOTable is imposing to have an
entry per FIELD, and thus the <FIELD ID=Exposure_Table>, which in our
case goes _inside_ the TABLE, has to have a dummy <TD> in your case as
it points to an outside table.
This might look like cosmetics thing, but in our view it reveals an
inconsistency between the VOTable real requirements and the way we would
use the VOTables in your example. Of course, this problem would be
solved by modifying somehow the DTD.
As a last example we were trying to use to clarify our own ideas, we
were identifying the structure we wanted to include in the SIAP to a
file system directory structure. In our approach, we send the directory
structure directly to the client, allowing it to display it the way it
wants. In your approach, a very simple directory structure would be
completely flat, with each subdirectory containing a pointer to the
directory where it hangs from, allowing to "join" directories and
subdirectories. Instead of giving the structure directly to the client,
you give it the tools to create the structure, but then it is the client
which has to do the whole process of "structurization". Although a valid
approach, we believe that the time spent by the client in doing this
should be spent by the server, which in the end is doing an asynchronous
work with the rest of the servers being requested.
Anyway, we believe this approach is much close to our idea of how things
should work, and we think we are working towards a positive evolution
for SIAP results.
Best regards,
P&J
--
Pedro Osuna Alcalaya
Software Engineer
VILSPA Archive Development Team
e-mail: Pedro.Osuna at esa.int
Tel + 34 91 8131314
European Space Agency
VILLAFRANCA Satellites Tracking Station
P.O. Box 50727
E-28080 Villafranca del Castillo
MADRID - SPAIN
On Mon, 2004-04-19 at 19:57, Francois Bonnarel wrote:
> Dear Jesus and Pedro,
>
> As I annouced in my mail last week-end I took your example
> and modified it a little bit by introducing in your peculiar fields
> "EXposure_Table" "Source_Table" and "Energy_Band_Table" a "VOTable
> ref" to a corresponding Table ID which one can find further in the
> VOTable document.
> In addition I created a "DIS:LowerLevelTable" pseudo UCD (DIS for
> discussion) which may be not necessary because the occurence of the
> reference may be sufficient to identify such kind of fields.
> I added the upper level Table Identifier field (ObsId) in the Exposure
> one in order to joint the Observation and Exposure Tables.
> Something similar would have to be done in Source and Energy_Band
> Tables which I was to lazy to complete!
> If I didn't misunderstood your example I Think this is a serialization
> of your data model at page 2, using some of the rules we described with
> Mireille in our "Data Model serialization in VOTable" note.
> This is done without modifying the VOTable DTD, which I think is reasonable
> for Boston, because there has been a very long process of discussion for
> version 1.1.
>
> Cheers
> François
>
>
>
> <?xml version = "1.0" ?>
> <!DOCTYPE VOTABLE (View Source for full doctype...)>
> <VOTABLE version = "1.0">
> <RESOURCE type="results">
> <DESCRIPTION> XMM-Newton Simple Image Protocol (SIAP) Service</DESCRIPTION>
> <INFO name="QUERY_STATUS" value="OK" />
> <TABLE ID="Observation">
> <FIELD ID="ObsId" ucd="OBS_ID" datatype="char" arraysize="*" />
> <FIELD ID="Target_Name" ucd="OBS_ID" datatype="char" arraysize="*" />
> <FIELD ID="Start_Time" ucd="VOX:OBS_START_TIME" datatype="char" arraysize="*" />
> <FIELD ID="End_Time" ucd="VOX:OBS_END_TIME" datatype="char" arraysize="*" />
> <FIELD ID="On_Time" ucd="VOX:OBS_DURATION" datatype="int" />
> <FIELD ID="RA" ucd="POS_EQ_RA_MAIN" datatype="char" arraysize="*" />
> <FIELD ID="DEC" ucd="POS_EQ_DEC_MAIN" datatype="char" arraysize="*" />
> <FIELD ID="FOV" ucd="VOX:Field_Of_View" datatype="char" arraysize="*" />
> <FIELD ID="FORMAT" ucd="VOX:Image_Format" datatype="char" arraysize="*" />
> <FIELD ID="Reference" ucd="DATA_LINK" datatype="char" arraysize="*" />
> <FIELD ID="Exposure_Table" ucd="DIS:LowerLevelTable" datatype="char" arraysize="*" ref= "Exposure" />
> <DATA>
> <TABLEDATA>
> <TR>
> <TD>0102640101</TD>
> <TD>XMM EPIC Target M33_1 </TD>
> <TD>2000-08-04 05:16:00.0 </TD>
> <TD>2000-08-04 10:27:12.0 </TD>
> <TD>18672</TD>
> <TD>23.458305</TD>
> <TD>30.66414</TD>
> <TD>0.72x0.72</TD>
> <TD>image/fits</TD>
> <TD>
> <![CDATA[http://xsa.vilspa.esa.es:8080/alo/jsp?obsno=0102640101&name=OIMAGE&level=PPS&extension=FTZ&protocol=HTTP]]>
> </TD>
> <TD>Exposure_Table</TD>
> </TR>
> </TABLEDATA>
> </DATA>
> </TABLE>
> <TABLE ID="Exposure">
> <FIELD ref="ObsId" />
> <FIELD ID="ExposureNumber" ucd="EXP_ID" datatype="char" arraysize="*" />
> <FIELD ID="Image_Name" ucd="VOX:Image_Title" datatype="char" arraysize="*" />
> <FIELD ID="Instrument" ucd="VOX:INST_ID" datatype="char" arraysize="*" />
> <FIELD ref="Start_Time" />
> <FIELD ref="End_Time" />
> <FIELD ref="On_Time" />
> <FIELD ref="RA" />
> <FIELD ref="DEC" />
> <FIELD ref="FOV" />
> <FIELD ref="FORMAT" />
> <FIELD ref="Reference" />
> <FIELD ID="Source_Table" ucd="DIS:LowerLevelTable" datatype="char" arraysize="*" ref= "Source" />
> <FIELD ID="Energy_Band_Table" ucd="DIS:LowerLevelTable" datatype="char" arraysize="*" ref= "Energy_Band" />
> <DATA>
> <DATA>
> <TABLEDATA>
> <TR>
> <TD>0102640101</TD>
> <TD>S001</TD>
> <TD>XMM EPIC EPN Image. Target : M31Core . Exposure S001 </TD>
> <TD>2000-08-04 06:06:43.0 </TD>
> <TD>2000-08-04 09:51:52.0 </TD>
> <TD>13509</TD>
> <TD>23.458305</TD>
> <TD>30.66414</TD>
> <TD>0.72x0.72</TD>
> <TD>image/fits</TD>
> <TD>
> <![CDATA[http://xsa.vilspa.esa.es:8080/alo/jsp?obsno=0102640101&&instname=PN&datasubsetno=8&name=IMAGE&level=PPS&extension=FTZ&protocol=HTTP]]>
> </TD>
> <TD>Source_Table</TD>
> <TD>Energy_Band_Table</TD>
> </TR>
> </TABLEDATA>
> </DATA>
> </TABLE>
> <TABLE ID = "Source">
> .......
> </TABLE>
> <TABLE ID = "Energy_Band">
> ......
> </TABLE>
> SAUVONS LA RECHERCHE : <http://recherche-en-danger.apinc.org/>
>
> =====================================================================
> Francois Bonnarel Observatoire Astronomique de Strasbourg
> CDS (Centre de donnees 11, rue de l'Universite
> astronomiques de Strasbourg) F--67000 Strasbourg (France)
>
> Tel: +33-(0)3 90 24 24 11 WWW: http://cdsweb.u-strasbg.fr/people/fb.html
> Fax: +33-(0)3 90 24 24 25 E-mail: bonnarel at astro.u-strasbg.fr
> ---------------------------------------------------------------------
--
Pedro Osuna Alcalaya
Software Engineer
VILSPA Archive Development Team
e-mail: Pedro.Osuna at esa.int
Tel + 34 91 8131314
European Space Agency
VILLAFRANCA Satellites Tracking Station
P.O. Box 50727
E-28080 Villafranca del Castillo
MADRID - SPAIN
More information about the dm
mailing list