Quantity "tables" - schema snippets
Martin Hill
mchill at dial.pipex.com
Mon Jan 26 10:52:27 PST 2004
We were looking at defining just these things, unfortunately I think we got
stuck on Quantity some time ago.
Some good attempts have been made from the other direction, in support of ADQL,
by (I assume?) Wil O Mullane et al, for co-ordinates and regions:
http://hea-www.harvard.edu/~arots/nvometa/region.xsd
http://hea-www.harvard.edu/~arots/nvometa/stc.xsd
Personally I would like to see more of these, rather than trying to squeeze all
and sundry into a generic Quantity model, but then I've said this before (I do
go on and on don't I).
One of the reasons is in Ed's example below. Using a tag <Quantity> and
specifying its type as an attribute makes it hard to validate using small schema
snippets (such as those above). It also makes it hard for XML UI tools to
interpret how to present choices to the user. It's much easier for all
concerned to have, using mostly the same data:
<StellarCatalogue>
<Galaxy name="NGC 300">
<Altname>PGC 3238</Altname>
<Position>
<WCS epoxinoxthingy="J2000">
<RA unit=sexigesimal>00:54:52.6</RA>
<DEC unit=sexigesmial>-37:40:57</DEC>
</WCS>
<Galactic>
<Long unit=degree>299.2306</Long>
<Lat unit=degree>-79.4210</Lat>
</Galactic>
</Position>
<Brightness band="B" unit="johnsonmag">8.95</Brightness>
<Brightness filter="Peach" unit="johnsonmag">6.95</Brightness>
<Extinction band="I" unit="johnsonmag">0.02</Extinction>
<Extinction band="B" unit="johnsonmag">0.055</Extinction>
<Shape>
<Ratio>0.73</Ratio>
<MorphologyT>7</MorphologyT>
<MorphologyM>SA(s)d</MorphologyM>
</Shape>
:
</Galaxy>
<Filter ID="peach">
<CenterWlen units='nm'>650</CenterFreq>
<FreqWidth units='millihertzteehee'>12</FreqWidth>
</Filter>
</StellarCatalogue>
and perhaps there should be more errors in there :-) And this is just an
example of using 'rich' XML, not of how I would actually expect to describe each
component.
See how clean it is. See how easy it is to read and write and understand even
for a human. See how easy it would be to write an XPath/XQuery even for a
beginner (not that you can't on the generic Quantity elements, but they would be
more obscure). See how easy it would be to split up into little bits to be
argued about independently instead of all at once - and build proper schema
snippets (WCS, Galactic, Brightness, Filter, etc) that describe exactly how a
component can be described. That we can then develop against.
And as for the overall schema, we can include everything we can think of for the
moment, as optional. So we don't actually have to agree it at all...
Cheers,
Martin
Tony Linde wrote:
> Interesting cross-post, Ed. I was just thinking today that one of the
> problems we're facing is that we have at least three groups (DM, VOTable and
> Registry) needing to represent the contents of set(s) of data for different
> reasons and all coming up with different solutions - potential for a real
> mess brewing. What we need is a DM for astro data which is representable as
> a schema which can be used to describe the contents of datasets for resource
> entries in the registry and also for representing the contents of the
> results of a query (for transporting in a VOTable). Oh, and VOQL group will
> also have an interest for constructing queries.
>
> Any VOTable does have to be hierarchical in nature: we may only be
> constructing simple queries now but future queries will have to cope with
> hierarchical results.
>
> I'm not sure that a generic flattener is feasible. It will probably work if
> you want to dump a hierarchical dataset into a *new* tabular form but would
> be difficult to map from the hierarchical results of a query into the (also
> hierarchical) table structure of an existing relational db since different
> people will make different assumptions when creating such a structure.
>
> Your example is fascinating but I wonder if someone else approaching the
> same set of data would create the same hierarchical structure: ie, how easy
> is it going to be to get all astronomers to agree on the one hierarchical
> structure for the representation of astronomical data. Maybe we need to
> define all the primitives - quantity, ucd, unit, instrument, etc - and some
> of the most common and agreed simple structures like position and then come
> up with the ability for astronomers, data centre managers, developers to
> combine these into appropriate hierarchies for representing metadata. Using
> the same primitives will at least allow interpretation of results even if
> they exist in unfamiliar structures.
>
> Cheers,
> Tony.
>
>
>>-----Original Message-----
>>From: owner-dm at eso.org [mailto:owner-dm at eso.org] On Behalf Of Ed Shaya
>>Sent: 22 January 2004 15:56
>>To: dm at ivoa.net
>>Subject: Quantity "tables"
>>
>>
>>This is a cross-post with VOTable.
>>
>>Simply put XML is hierarchical and a table is not. Hence,
>>there is this misfit between XML tools (XQuery, XPath, XSLT,
>>WebServices) and our XML/Table efforts (Astrores, VOTable,
>>XDF, Dataset, etc). XML's hierarchies make it more suitable
>>for describing an object's structure. Tables are compact and
>>human-friendly, especially when one gets to large (but not too
>>large) amounts of info. We want it all, but we are
>>struggling to make tables fit beautifully into XML, and
>>falling into circles of inconsistencies.
>>
>>In true XML, tables are something to convert to or from.
>>XHTML has tables explicitly described, but XHTML is a display
>>language. One transforms from your true XML into XHTML only
>>at display time.
>>
>>XML is meant to interchange with relational databases. Using
>>a schema based on true XML, one creates a database
>>configuration and flattens the documents into one or several
>>tables. The tables are now strictly speaking outside of the
>>XML realm but an XML front end allows one to address the data
>>as if it were still in true XML. Although, it should be
>>noted that native XML databases are getting pretty mature and
>>maybe we won't be doing this much longer.
>>
>>I conclude that we should start off with a true XML
>>description of data (ie, not tabular) and then after that
>>matures, rethink tabular formats, but as either a compressed
>>transmission mode or as a display mode or as the internals of
>>an object-relational database. That way, all tools interface
>>with the data in true XML fashions. If I am right, then it
>>would follow that even our image/table display tools like
>>Alladin and Mirage would benefit from a hierarchical view of data.
>>
>>Just to make clear what I mean by a "hierarchical view of
>>data", below is an example of the "non-tabular way". This is
>>just one galaxy and one must imagine that there are many in
>>this object oriented database. It would be easy now to
>>extend this to include additional metadata about each
>>measurement, who observed it, when, with which instrument,
>>etc. It would also be interested to create a little schema
>>for each Quantity/@type. A B_T_Quantity.xsd which extends
>>Quantity could substitute for the <Quantity type="B_T"/> and
>>provide suitable extensions and restrictions on both metadata
>>and datatypes.
>>
>>OK. Now the crux of this all is that the Tabular form of
>>this is done by a auto-flattener that simply turns this into
>>a series of tables exactly as an XML to DB converter does.
>>Now you have a compact form to transfer and one that can go
>>right into your local/private DB. If it is an object
>>relational DB you should be able to access the data with the
>>original true XML schema and XPATH or XQuery.
>>
>>======================================================
>><AstroObject type="galaxy" name="NGC 300">
>> <Quantity type="altname" name="PGC
>>Num"><value>3238</value><units><unitless/></units></Quantity>
>> <Group name="position">
>> <Quantity type="RA
>>
>>
>>J2000"><value>00:54:52.6</value><units><unit>xs:time</unit></u
>>nits></Quantity>
>> <Quantity type="DE
>>J2000"><value>-37:40:57</value><units><unit>sexigesimal</unit>
>></units></Quantity>
>> </Group>
>> <Quantity type="Galactic_long"
>>name="l"><value>299.2306</value><units><unit>degree</unit></un
>>its></Quantity>
>> <Quantity type="Galactic_lat"
>>name="b"><value>-79.4210</value><units><unit>degree</unit></un
>>its></Quantity>
>> <Quantity
>>type="SGL"><value>259.8113</value><units><unit>degree</unit></
>>units></Quantity>
>> <Quantity
>>type="SGB"><value>-9.4984</value><units><unit>degree</unit></u
>>nits></Quantity>
>> <Quantity type="brightness absolute B total"
>>name="B_T"><value>8.95</value><units><unit>magnitude</unit></u
>>nits></Quantity>
>> <Quantity type="extinction I-band"
>>name="A_I"><value>0.02</value><units><unit>magnitude</unit></u
>>nits></Quantity>
>> <Quantity type="extinction B-band"
>>name="A_B"><value>0.055</value><units><unit>magnitude</unit></
>>units></Quantity>
>> <Quantity type="axis ratio b/a" shape
>>name="b/a"><value>0.73</value><unitless/></Quantity>
>> <Quantity type="morphology type T"
>>name="Ttype"><value>7</value><unitless/></Quantity>
>> <Quantity type="morphology type M"
>>name="Mtype"><value>SA(s)d</value><unitless/></Quantity>
>> <Group type="velocity radial">
>> <Quantity type="geocentric" name="V_gsr">
>> <value errorValue="4." >101.</value>
>> <units><unit>km</unit><unit power="-1">s</unit></units>
>> </Quantity>
>> <Quantity type="heliocentric" name="V_helio">
>> <value errorValue="4.">144.</value>
>> <units><unit>km</unit><unit power="-1">s</unit></units>
>> </Quantity>
>> </Group>
>> <Quantity type="linewidth 21cm @20percentOfPeak" name="W20">
>> <value ErrorValue="7.">166.</value>
>> <units><unit>km</unit><unit power="-1">s</unit></units>
>> </Quantity>
>> <Quantity type="distance Cepheid" name="Cepheid dist"><value
>>errorVAlue="2.3"
>>
>>>26.66</value><units><unit>Mpc</unit></units></Quantity>
>>
>> <Quantity type="distance TRGB" name="TRGB
>>dist"><value>UNKNOWN</value><units><unit>Mpc</unit></units></Quantity>
>> <Quantity type="distance PNLF" name="PNLF
>>dist"><value>26.9</value><units><unit>Mpc</unit></units></Quantity>
>> <Quantity type="distance SBF" name="SBF
>>dist"><value>UNKNOWN</value><units><unit>Mpc</unit></units></Quantity>
>> <Quantity type="distance TF" name="Tully-Fisher
>>dist"><value>UNKNOWN</value><units><unit>Mpc</unit></units></Quantity>
>> <Quantity type="axes"><valueList>21.9
>>15.5</valueList><units><unit>arcmin</unit></units></Quantity>
>> <photometry>
>> <region>
>> <fitsFile href="HSTDA/retrieve/n300.u65w0201r.fits">
>> <instrument>WFPC2</instrument>
>> <observationDate>2001:05:06</observationDate>
>>
>><rightAscension>1.372291666667E+01</rightAscension>
>> <declination>-3.768333333333E+01</declination>
>> <rotation>168.502</rotation>
>> <filters>F814W</filters>
>> <exposureTime units="second">40.</exposureTime>
>> <proposalId>8599</proposalId>
>> <observer>Boeker</observer>
>> <history>
>> <date>2001:08:24</date>
>> <creator>Felicia Tam</creator>
>> <item>mosaic of all 4 ccds</item>
>> </history>
>> </fitsFile>
>> <fitsFile href="HSTDA/retrieve/n300.u65w0202r.fits">
>> <telescope>HST</telescope>
>> <instrument>WFPC2</instrument>
>> <observationDate>2001:05:06</observationDate>
>>
>><rightAscension>1.372291666667E+01</rightAscension>
>> <declination>-3.768333333333E+01</declination>
>> <rotation>168.502</rotation>
>> <filters>F814W</filters>
>> <exposureTime units="second">300.</exposureTime>
>> <proposalId>8599</proposalId>
>> <observer>Boeker</observer>
>> <history>
>> <date>2001:08:24</date>
>> <creator>Felicia Tam</creator>
>> <item>mosaic of all 4 ccds</item>
>> </history>
>> </fitsFile>
>> <fitsFile href="HSTDA/retrieve/n300.u65w0203r.fits">
>> <telescope>HST</telescope>
>> <instrument>WFPC2</instrument>
>> <observationDate>2001:05:06</observationDate>
>>
>><rightAscension>1.372291666667E+01</rightAscension>
>> <declination>-3.768333333333E+01</declination>
>> <rotation>168.502</rotation>
>> <filters>F814W</filters>
>> <exposureTime units="second">300.</exposureTime>
>> <proposalId>8599</proposalId>
>> <observer>Boeker</observer>
>> <history>
>> <date>2001:08:24</date>
>> <creator>Felicia Tam</creator>
>> <item>mosaic of all 4 ccds</item>
>> </history>
>> </fitsFile>
>> </region>
>> <region>
>> <fitsFile href="shoko/n300.fits">
>> <telescope>HST</telescope>
>> <instrument>WFPC2</instrument>
>> <observationDate>2001:07:02</observationDate>
>>
>><rightAscension>1.375416666667E+01</rightAscension>
>> <declination>-3.758000000000E+01</declination>
>> <rotation>-154.402</rotation>
>> <filters>F814W</filters>
>> <exposureTime units="second">600.</exposureTime>
>> <proposalId>9162</proposalId>
>> <observer>Tully</observer>
>> <history>
>> <date>2001:07:09</date>
>> <creator>Shoko Sakai</creator>
>> <item>mosaic of all 4 ccds</item>
>> <item>cosmic ray cleaned and summed</item>
>> </history>
>> </fitsFile>
>> <psFile name="NGC 300 Color-Magnitude Diagram"
>>href="shoko/n300cmd.ps"/>
>> </region>
>> </photometry>
>> <kinematics>
>> <reference>
>> <title>H I studies of the Sculptor group
>>galaxies. VI - NGC
>>300</title>
>>
>><author><initial>D</initial><lastName>Puche</lastName></author>
>>
>> <author><initial>C</initial><lastName>Carignan</lastName></author>
>>
>><author><initial>A</initial><lastName>Bosma</lastName></author>
>> <abstract>A study of the kinematics of the
>>Sculptor group galaxy NGC 300 is presented. VLA observations
>>of the H I line show a dramatically warped H I disk just
>>outside the optical disk. A two-component mass model, fitted
>>to the data, gives a total mass of 2.4E10 solar masses at the
>>last observed point of the rotation curve. This value implies
>>an (M/L_B)global about 11 solar masses/solar luminosity for
>>the galaxy which is compared with the total dynamical
>>(M/L_B)dyn of the group. This comparison suggests that dark
>>matter is probably more concentrated around the galaxies of
>>the group than uniformly distributed in the intergalactic medium.
>> </abstract>
>> <bibcode>1990AJ....100.1468P</bibcode>
>> </reference>
>> </kinematics>
>></galaxy>
>>
>>
>>Ed
>>
>>
>
>
>
>
--
Martin Hill
Software Engineer
AstroGrid @ ROE
Tel: +44 7901 55 24 66
www.astrogrid.org
More information about the dm
mailing list