UCD validator
Bo Milvang-Jensen
milvang at astro.ku.dk
Tue Mar 1 18:12:55 CET 2022
Dear Mireille and Mark,
(I added Laura Mascetti in cc)
Thank you for this! I am a fan of STILTS, but I had to download the
latest version to get votlint validator that accepted the ucd=true
option. I could then run it as follows, first converting my FITS table
to a VOTable, and then running votlint:
java -jar stilts.jar tpipe in=COSMOS2020_CLASSIC_R1_v2.0.1_p3_test1.fits
ofmt=votable out=COSMOS2020_CLASSIC_R1_v2.0.1_p3_test1.xml cmd="rowrange
1 1"
java -jar stilts.jar -verbose votlint ucd=true validate=true
votable=COSMOS2020_CLASSIC_R1_v2.0.1_p3_test1.xml
On my table with 754 columns it gave this result:
INFO: votlint version=null validate=true ucd=true unit=null maxrepeat=4
votable=COSMOS2020_CLASSIC_R1_v2.0.1_p3_test1.xml out=-
WARNING (l.31, c.93): Bad VOUnit "pix2" (BAD_SYNTAX): unity parser error
at character 4: syntax error
WARNING (l.34, c.93): Bad VOUnit "pix2" (BAD_SYNTAX): unity parser error
at character 4: syntax error
WARNING (l.37, c.93): Bad VOUnit "pix2" (BAD_SYNTAX): unity parser error
at character 4: syntax error
ERROR (l.1783, c.90): Bad UCD "stat.max;phot.mag.sb;em.opt.I"
(BAD_SEQUENCE): UCD starting with a SECONDARY UCD word: "stat.max"! Such
words can NOT be in first position.
WARNING (l.1888, c.81): Bad VOUnit "dex(solMass)" (UNKNOWN_UNIT):
Unknown unit "Solar mass"
INFO (maxrepeat=4) suppressing more messages like "Bad VOUnit
"dex(solMass)" (UNKNOWN_UNIT): Unknown unit "Solar mass""
That single UCD error is clear (my mistake). So I have written 753 valid
UCDs :-)
Interesting with the comments on the units (units which I understand
have been produced/formatted by astropy). I guess pix2 has to be pixel2
or pixel**2. In my team we have been debating whether dex is valid (or
should be log), but votlint does not complain. But is solMass wrong?
Mark, I have not yet tried what you suggested. Would it be a more
comprehensive test suite than votlint?
Kind regards, Bo
On 3/1/22 6:01 PM, Mark Taylor wrote:
> Hi all,
>
> stilts/topcat can do this, but votlint is not the easiest way to access
> the UCD validation functionality, the ucdStatus and ucdMessage
> functions in the expression language are the best tools for the job:
>
> http://www.starlink.ac.uk/stilts/sun256/uk.ac.starlink.ttools.func.VO.html
>
> These are based on Gregory Mantelet's Ucidy library which does quite
> careful UCD validation.
>
> I just sent Alberto separately an example of how this can be used.
> I guess he will forward it to Bo(?) but I can repeat it here if required.
>
> Mark
>
>
> On Tue, 1 Mar 2022, Mireille LOUYS wrote:
>
>> Dear Bo, dear Alberto,
>>
>> the tool you are using for validation does not handle properly contatenation
>> for 3 terms especially as you describe when two SS need to be concatenated.
>>
>> we use instead this assigning tool now :
>> http://cds.u-strasbg.fr/UCD/cgi-bin/descr2ucd
>>
>> from the definition : "luminosity in V band min value"
>> it builds : /phys.luminosity;em.opt.V;stat.min/
>>
>> if you want to validate your ucd fields, I think the stilts tool from Mark
>> Taylor will help a lot
>> http://www.star.bristol.ac.uk/~mbt/stilts/sun256/votlint-usage.html
>>
>> here attached is a test votable :
>> when you install stilts and run this command :
>> />> stilts -verbose votlint ucd=true validate=true votable=./testucd.xml
>> you get this result
>> /INFO: votlint version=null validate=true ucd=true unit=null maxrepeat=4
>> votable=./testucd.xml out=-
>> ERROR (l.22, c.97): Bad UCD "em.opt.V;phys.luminosity;stat.min"
>> (BAD_SEQUENCE): UCD starting with a SECONDARY UCD word: "em.opt.V"! Such words
>> can NOT be in first position.
>>
>> so the 2 SS combinations are valid and only em.opt is detected in the wrong
>> position , which is correct.
>>
>> I encourage you to use votlint for your checks.
>>
>> best , Mireille
>>
>> Le 25/02/2022 à 16:59, Bo Milvang-Jensen a écrit :
>>> Dear Alberto and all,
>>>
>>> Thank you for adding me in cc. I am the user who is working on assigning
>>> UCDs to 1200 catalogue columns, prior to publication in ESO's Phase 3 system
>>> (supported by Laura Mascetti), and that is a bit of a headache. Just to
>>> spell out this particular problem: the UCD1+ v1.4 document only says (in my
>>> reading) that an S word cannot be first, it does not say that an S word has
>>> to be last! but the validator at CDS seems to think so.
>>>
>>> This means that according to the validator at CDS there is no way to
>>> construct a UCD describing a minimum value of a V-band luminosity, as the
>>> corresponding 3 words cannot be combined in any way:
>>>
>>> phys.luminosity;stat.min;em.opt.V --- violates the claimed ESS rule
>>> phys.luminosity;em.opt.V;stat.min --- violates the claimed ESS rule
>>> stat.min;phys.luminosity;em.opt.V --- of course not allowed (stat.min: S
>>> word)
>>>
>>> The validator will also complain about the sequences QSS and PSS.
>>>
>>> PS: I have a couple of other questions/problems with the UCD standard that I
>>> would like to ask/report. What is the best channel?
>>>
>>> Kind regards, Bo
>>>
>>> On 2/25/22 4:31 PM, Alberto Micol wrote:
>>>> Dear CDS and UCD editors,
>>>>
>>>> I'd like to follow up on two original inquiries of mine, both sent
>>>> 21.02.2020, attached below,
>>>>
>>>> and ask two questions.
>>>>
>>>> We currently have an inquiry from one of our ESO principal investigators
>>>> (in cc),
>>>>
>>>> who needs to identify the correct UCDs needed to publish a catalog,
>>>>
>>>> and his inquiry is exactly on the same issue, the CDS validator, published
>>>> at:
>>>>
>>>> http://cdsweb.u-strasbg.fr/UCD/tools.htx
>>>>
>>>> Using your "validate" form in that page on the following UCD:
>>>>
>>>> phot.flux;em.radio;arith.ratio
>>>>
>>>> one gest the error:
>>>>
>>>> 8 **** invalid sequence of syntax codes (ESS)
>>>>
>>>> Same result for BO's UCD:
>>>>
>>>> phys.luminosity;stat.min;em.opt.V
>>>>
>>>> 8 **** invalid sequence of syntax codes (ESS)
>>>>
>>>> And similar result for my UCD (see first attachment):
>>>>
>>>> phot.flux.density;em.wl;stat.uncalib;src.net
>>>>
>>>> where I get:
>>>>
>>>> 8 **** invalid sequence of syntax codes (EQSS)
>>>>
>>>> But if I remove any of the two "S" words, than I no longer get an error,
>>>>
>>>> despite the fact that a EQS should be equivalent to an ESS, given that Q
>>>> can be either a P or an S.
>>>>
>>>> So, it seems to me that your validator does not accept 2 "S" words in a
>>>> UCD.
>>>>
>>>> Please note that I took that first UCD above
>>>> (phot.flux;em.radio;arith.ratio)
>>>>
>>>> from a list of examples of *legal UCDs* in the UCD standard itself:
>>>>
>>>> see https://www.ivoa.net/documents/REC/UCD/UCD-20050812.html#toc-header-10
>>>>
>>>> So, that must be a valid UCD.
>>>>
>>>> *Q1: Could you please verify the logic of the validator, also considering
>>>> what follows?*
>>>>
>>>> Before concluding, let me add a potential explanation of this issue…
>>>>
>>>> Probably this is the result of a statement in the IVOA endorsed note, par.
>>>> D2, at:
>>>>
>>>> https://www.ivoa.net/documents/UCD1+/20210616/EN-UCDlist-1.4-20210616.html#tth_sEcD.2
>>>>
>>>> which states that:
>>>>
>>>> "P, S and Q are the labels expressing in which position of a UCD
>>>> expression a term can be used,
>>>>
>>>> P in first place, S as suffix, and Q in both allowed position: head and
>>>> tail.
>>>>
>>>> "
>>>>
>>>> The word "tail" is probably taken literally by your validator, resulting
>>>> in this 2 "S" words-error.
>>>>
>>>> But it is that endorsed note that uses "tail" in a wrong way,
>>>>
>>>> as explained in my second attachment (see therein: "contradiction with the
>>>> sentence in B").
>>>>
>>>> *Q2: I'd like to ask the UCD editors to please consider my suggested
>>>> amendment to the standard (see second attachment),*
>>>>
>>>> *So to avoid any possible confusion on the word "tail".*
>>>>
>>>> Many thanks, and sorry for the long email,
>>>>
>>>> Alberto
>>>>
>>>> *From: *Alberto Micol <amicol at eso.org>
>>>> *Date: *Friday, 21. February 2020 at 10:12
>>>> *To: *CDS question <cds-question at unistra.fr>
>>>> *Cc: *Laura Mascetti <lmascett at partner.eso.org>, Baptiste Cecconi
>>>> <baptiste.cecconi at obspm.fr>
>>>> *Subject: *UCD validator
>>>>
>>>> Using the UCD validator service:
>>>>
>>>> http://cds.u-strasbg.fr/UCD/cgi-bin/WS_validate
>>>>
>>>> on the following ucd1+:
>>>>
>>>> phot.flux.density;em.wl;stat.uncalib;src.net
>>>>
>>>> I get the following error:
>>>>
>>>> 8 **** invalid sequence of syntax codes (EQSS)
>>>>
>>>> It is unclear to me what the problem is, as the em.wl (Q) is immediately
>>>> following the photometric part (E),
>>>>
>>>> and the two secondary (S) tokens are indeed at the end of the ucd.
>>>>
>>>> Could you let me know what the problem is, and then maybe modify the
>>>> validator to provide a more explanatory text,
>>>>
>>>> as it does, for example, when inputting:
>>>>
>>>> phot.flux.density;stat.uncalib;src.net;em.wl
>>>>
>>>> whose error is clearly telling the user what to repair:
>>>>
>>>> 8 **** invalid sequence of syntax codes (ESSQ) S word must be last
>>>>
>>>> Many thanks,
>>>>
>>>> Alberto
>>>>
>>>> PS: I understand that the UCD standard gives contradictory information
>>>> regarding the Q tokens, I will write to Baptiste to clarify that.
>>>>
>>>> *From: *Alberto Micol <amicol at eso.org>
>>>> *Date: *Friday, 21. February 2020 at 10:36
>>>> *To: *Baptiste Cecconi <baptiste.cecconi at obspm.fr>
>>>> *Cc: *Laura Mascetti <lmascett at partner.eso.org>
>>>> *Subject: *UCD1+: some confusion on the "Q" tokens
>>>>
>>>> Ciao Baptiste,
>>>>
>>>> I'd like to highlight to you a (slight) inconsistency in the UCD1+
>>>> standard (v1.3), but also repeated in the current Proposed Endorsed Note
>>>> 2020-02-12v1.4.
>>>>
>>>> In: A.2 P or S syntax code: Which is the most pertinent position for a UCD
>>>> word?
>>>>
>>>> /P, S and Q are the labels expressing in which position of a UCD
>>>> expression a term can be used, P in first place, S as suffix, and Q in
>>>> both allowed position: head and tail. /
>>>>
>>>> The issue I have is with "… in both allowed position(s): head and tail",
>>>> which seems to indicate that a Q term
>>>>
>>>> can either be first or last, but not in the middle of a longer ucd, and
>>>> this would be in contradiction with the sentence in B:
>>>>
>>>> B List of valid words
>>>>
>>>> 3. /“Q” means that the word can be used indifferently as first or
>>>> secondary word; /
>>>>
>>>> Can I suggest to modify that sentence in A2, as follows:
>>>>
>>>> /P, S and Q are the labels expressing in which position of a UCD
>>>> expression a term can be used, P in first place, S as suffix, and Q
>>>> //either as P or S./
>>>>
>>>> ?
>>>>
>>>> Many thanks,
>>>>
>>>> Alberto
>>>>
>> --
>> --
>> Mireille Louys, MCF (Associate Professor)
>> Centre de données CDS IPSEO, Images, Laboratoire Icube
>> Observatoire de Strasbourg Telecom Physique Strasbourg
>> 11 rue de l'Université 300, Bd Sebastien Brandt CS 10413
>> F- 67000-STRASBOURG F-67412 ILLKIRCH Cedex
>> Tel: +33 3 68 85 24 34
>>
> --
> Mark Taylor Astronomical Programmer Physics, Bristol University, UK
> m.b.taylor at bristol.ac.uk http://www.star.bristol.ac.uk/~mbt/
More information about the semantics
mailing list