UCD validator
Mark Taylor
m.b.taylor at bristol.ac.uk
Thu Mar 3 11:03:07 CET 2022
Mireille,
indeed, if your UCDs are already in a VOTable (FIELDs or PARAMs)
then votlint is a good way to validate them. For your attached example:
% stilts votlint testucd.xml
ERROR (l.22, c.97): Bad UCD "em.opt.V;phys.luminosity;stat.min" (BAD_SEQUENCE): UCD starting with a SECONDARY UCD word: "em.opt.V"! Such words can NOT be in first position.
And in fact it seems that this was suitable for Bo as well.
The reason I suggested using the ucdStatus and ucdMessage functions
was because Alberto had asked me (private email, not this thread):
On Tue, 1 Mar 2022, Alberto Micol wrote:
> I'd like to know if there is an easy way to validate a simple ascii
> list of UCDs.
>
> Do you know a tool that can do that, without having to build a full
> VOTable (votlint would then validate the ucds therein, but it is of
> course cumbersome to have to build a votable just to validate a list
> of ucds…) ?
The actual validation code is the same in both cases, and performed
by Gregory Mantelet's Ucidy library: https://github.com/gmantele/ucidy/.
So if topcat/stilts is not convenient, an alternative would be to write
your own code harnessing Ucidy.
Please let me know if you need any more information or help on this.
Mark
On Thu, 3 Mar 2022, Mireille LOUYS wrote:
> Hi Mark ,
> thanks for your explanation.
>
> I am interested in these checking tools , in order to manage the evolution of
> UCD lists.
> Do you mind propagating your exemple ?
>
> my test table ( attached ) has only params , and no data table so the function
> I found in the last topcat version cannot
> work.
>
> thanks , Mireille
>
>
>
> Le 01/03/2022 à 18:01, Mark Taylor a écrit :
> > Hi all,
> >
> > stilts/topcat can do this, but votlint is not the easiest way to access
> > the UCD validation functionality, the ucdStatus and ucdMessage
> > functions in the expression language are the best tools for the job:
> >
> > http://www.starlink.ac.uk/stilts/sun256/uk.ac.starlink.ttools.func.VO.html
> >
> > These are based on Gregory Mantelet's Ucidy library which does quite
> > careful UCD validation.
> >
> > I just sent Alberto separately an example of how this can be used.
> > I guess he will forward it to Bo(?) but I can repeat it here if required.
> >
> > Mark
> >
> >
> > On Tue, 1 Mar 2022, Mireille LOUYS wrote:
> >
> > > Dear Bo, dear Alberto,
> > >
> > > the tool you are using for validation does not handle properly
> > > contatenation
> > > for 3 terms especially as you describe when two SS need to be
> > > concatenated.
> > >
> > > we use instead this assigning tool now :
> > > http://cds.u-strasbg.fr/UCD/cgi-bin/descr2ucd
> > >
> > > from the definition : "luminosity in V band min value"
> > > it builds : /phys.luminosity;em.opt.V;stat.min/
> > >
> > > if you want to validate your ucd fields, I think the stilts tool from Mark
> > > Taylor will help a lot
> > > http://www.star.bristol.ac.uk/~mbt/stilts/sun256/votlint-usage.html
> > >
> > > here attached is a test votable :
> > > when you install stilts and run this command :
> > > />> stilts -verbose votlint ucd=true validate=true votable=./testucd.xml
> > > you get this result
> > > /INFO: votlint version=null validate=true ucd=true unit=null maxrepeat=4
> > > votable=./testucd.xml out=-
> > > ERROR (l.22, c.97): Bad UCD "em.opt.V;phys.luminosity;stat.min"
> > > (BAD_SEQUENCE): UCD starting with a SECONDARY UCD word: "em.opt.V"! Such
> > > words
> > > can NOT be in first position.
> > >
> > > so the 2 SS combinations are valid and only em.opt is detected in the
> > > wrong
> > > position , which is correct.
> > >
> > > I encourage you to use votlint for your checks.
> > >
> > > best , Mireille
> > >
> > > Le 25/02/2022 à 16:59, Bo Milvang-Jensen a écrit :
> > > > Dear Alberto and all,
> > > >
> > > > Thank you for adding me in cc. I am the user who is working on assigning
> > > > UCDs to 1200 catalogue columns, prior to publication in ESO's Phase 3
> > > > system
> > > > (supported by Laura Mascetti), and that is a bit of a headache. Just to
> > > > spell out this particular problem: the UCD1+ v1.4 document only says (in
> > > > my
> > > > reading) that an S word cannot be first, it does not say that an S word
> > > > has
> > > > to be last! but the validator at CDS seems to think so.
> > > >
> > > > This means that according to the validator at CDS there is no way to
> > > > construct a UCD describing a minimum value of a V-band luminosity, as
> > > > the
> > > > corresponding 3 words cannot be combined in any way:
> > > >
> > > > phys.luminosity;stat.min;em.opt.V --- violates the claimed ESS rule
> > > > phys.luminosity;em.opt.V;stat.min --- violates the claimed ESS rule
> > > > stat.min;phys.luminosity;em.opt.V --- of course not allowed (stat.min: S
> > > > word)
> > > >
> > > > The validator will also complain about the sequences QSS and PSS.
> > > >
> > > > PS: I have a couple of other questions/problems with the UCD standard
> > > > that I
> > > > would like to ask/report. What is the best channel?
> > > >
> > > > Kind regards, Bo
> > > >
> > > > On 2/25/22 4:31 PM, Alberto Micol wrote:
> > > > > Dear CDS and UCD editors,
> > > > >
> > > > > I'd like to follow up on two original inquiries of mine, both sent
> > > > > 21.02.2020, attached below,
> > > > >
> > > > > and ask two questions.
> > > > >
> > > > > We currently have an inquiry from one of our ESO principal
> > > > > investigators
> > > > > (in cc),
> > > > >
> > > > > who needs to identify the correct UCDs needed to publish a catalog,
> > > > >
> > > > > and his inquiry is exactly on the same issue, the CDS validator,
> > > > > published
> > > > > at:
> > > > >
> > > > > http://cdsweb.u-strasbg.fr/UCD/tools.htx
> > > > >
> > > > > Using your "validate" form in that page on the following UCD:
> > > > >
> > > > > phot.flux;em.radio;arith.ratio
> > > > >
> > > > > one gest the error:
> > > > >
> > > > > 8 **** invalid sequence of syntax codes (ESS)
> > > > >
> > > > > Same result for BO's UCD:
> > > > >
> > > > > phys.luminosity;stat.min;em.opt.V
> > > > >
> > > > > 8 **** invalid sequence of syntax codes (ESS)
> > > > >
> > > > > And similar result for my UCD (see first attachment):
> > > > >
> > > > > phot.flux.density;em.wl;stat.uncalib;src.net
> > > > >
> > > > > where I get:
> > > > >
> > > > > 8 **** invalid sequence of syntax codes (EQSS)
> > > > >
> > > > > But if I remove any of the two "S" words, than I no longer get an
> > > > > error,
> > > > >
> > > > > despite the fact that a EQS should be equivalent to an ESS, given that
> > > > > Q
> > > > > can be either a P or an S.
> > > > >
> > > > > So, it seems to me that your validator does not accept 2 "S" words in
> > > > > a
> > > > > UCD.
> > > > >
> > > > > Please note that I took that first UCD above
> > > > > (phot.flux;em.radio;arith.ratio)
> > > > >
> > > > > from a list of examples of *legal UCDs* in the UCD standard itself:
> > > > >
> > > > > see
> > > > > https://www.ivoa.net/documents/REC/UCD/UCD-20050812.html#toc-header-10
> > > > >
> > > > > So, that must be a valid UCD.
> > > > >
> > > > > *Q1: Could you please verify the logic of the validator, also
> > > > > considering
> > > > > what follows?*
> > > > >
> > > > > Before concluding, let me add a potential explanation of this issue…
> > > > >
> > > > > Probably this is the result of a statement in the IVOA endorsed note,
> > > > > par.
> > > > > D2, at:
> > > > >
> > > > > https://www.ivoa.net/documents/UCD1+/20210616/EN-UCDlist-1.4-20210616.html#tth_sEcD.2
> > > > >
> > > > > which states that:
> > > > >
> > > > > "P, S and Q are the labels expressing in which position of a UCD
> > > > > expression a term can be used,
> > > > >
> > > > > P in first place, S as suffix, and Q in both allowed position: head
> > > > > and
> > > > > tail.
> > > > >
> > > > > "
> > > > >
> > > > > The word "tail" is probably taken literally by your validator,
> > > > > resulting
> > > > > in this 2 "S" words-error.
> > > > >
> > > > > But it is that endorsed note that uses "tail" in a wrong way,
> > > > >
> > > > > as explained in my second attachment (see therein: "contradiction with
> > > > > the
> > > > > sentence in B").
> > > > >
> > > > > *Q2: I'd like to ask the UCD editors to please consider my suggested
> > > > > amendment to the standard (see second attachment),*
> > > > >
> > > > > *So to avoid any possible confusion on the word "tail".*
> > > > >
> > > > > Many thanks, and sorry for the long email,
> > > > >
> > > > > Alberto
> > > > >
> > > > > *From: *Alberto Micol <amicol at eso.org>
> > > > > *Date: *Friday, 21. February 2020 at 10:12
> > > > > *To: *CDS question <cds-question at unistra.fr>
> > > > > *Cc: *Laura Mascetti <lmascett at partner.eso.org>, Baptiste Cecconi
> > > > > <baptiste.cecconi at obspm.fr>
> > > > > *Subject: *UCD validator
> > > > >
> > > > > Using the UCD validator service:
> > > > >
> > > > > http://cds.u-strasbg.fr/UCD/cgi-bin/WS_validate
> > > > >
> > > > > on the following ucd1+:
> > > > >
> > > > > phot.flux.density;em.wl;stat.uncalib;src.net
> > > > >
> > > > > I get the following error:
> > > > >
> > > > > 8 **** invalid sequence of syntax codes (EQSS)
> > > > >
> > > > > It is unclear to me what the problem is, as the em.wl (Q) is
> > > > > immediately
> > > > > following the photometric part (E),
> > > > >
> > > > > and the two secondary (S) tokens are indeed at the end of the ucd.
> > > > >
> > > > > Could you let me know what the problem is, and then maybe modify the
> > > > > validator to provide a more explanatory text,
> > > > >
> > > > > as it does, for example, when inputting:
> > > > >
> > > > > phot.flux.density;stat.uncalib;src.net;em.wl
> > > > >
> > > > > whose error is clearly telling the user what to repair:
> > > > >
> > > > > 8 **** invalid sequence of syntax codes (ESSQ) S word must be last
> > > > >
> > > > > Many thanks,
> > > > >
> > > > > Alberto
> > > > >
> > > > > PS: I understand that the UCD standard gives contradictory information
> > > > > regarding the Q tokens, I will write to Baptiste to clarify that.
> > > > >
> > > > > *From: *Alberto Micol <amicol at eso.org>
> > > > > *Date: *Friday, 21. February 2020 at 10:36
> > > > > *To: *Baptiste Cecconi <baptiste.cecconi at obspm.fr>
> > > > > *Cc: *Laura Mascetti <lmascett at partner.eso.org>
> > > > > *Subject: *UCD1+: some confusion on the "Q" tokens
> > > > >
> > > > > Ciao Baptiste,
> > > > >
> > > > > I'd like to highlight to you a (slight) inconsistency in the UCD1+
> > > > > standard (v1.3), but also repeated in the current Proposed Endorsed
> > > > > Note
> > > > > 2020-02-12v1.4.
> > > > >
> > > > > In: A.2 P or S syntax code: Which is the most pertinent position for a
> > > > > UCD
> > > > > word?
> > > > >
> > > > > /P, S and Q are the labels expressing in which position of a UCD
> > > > > expression a term can be used, P in first place, S as suffix, and Q in
> > > > > both allowed position: head and tail. /
> > > > >
> > > > > The issue I have is with "… in both allowed position(s): head and
> > > > > tail",
> > > > > which seems to indicate that a Q term
> > > > >
> > > > > can either be first or last, but not in the middle of a longer ucd,
> > > > > and
> > > > > this would be in contradiction with the sentence in B:
> > > > >
> > > > > B List of valid words
> > > > >
> > > > > 3. /“Q” means that the word can be used indifferently as first or
> > > > > secondary word; /
> > > > >
> > > > > Can I suggest to modify that sentence in A2, as follows:
> > > > >
> > > > > /P, S and Q are the labels expressing in which position of a UCD
> > > > > expression a term can be used, P in first place, S as suffix, and Q
> > > > > //either as P or S./
> > > > >
> > > > > ?
> > > > >
> > > > > Many thanks,
> > > > >
> > > > > Alberto
> > > > >
> > > --
> > > --
> > > Mireille Louys, MCF (Associate Professor)
> > > Centre de données CDS IPSEO, Images, Laboratoire Icube
> > > Observatoire de Strasbourg Telecom Physique Strasbourg
> > > 11 rue de l'Université 300, Bd Sebastien Brandt CS 10413
> > > F- 67000-STRASBOURG F-67412 ILLKIRCH Cedex
> > > Tel: +33 3 68 85 24 34
> > >
> > --
> > Mark Taylor Astronomical Programmer Physics, Bristol University, UK
> > m.b.taylor at bristol.ac.uk http://www.star.bristol.ac.uk/~mbt/
>
> --
> --
> Mireille Louys, MCF (Associate Professor)
> Centre de données CDS IPSEO, Images, Laboratoire Icube
> Observatoire de Strasbourg Telecom Physique Strasbourg
> 11 rue de l'Université 300, Bd Sebastien Brandt CS 10413
> F- 67000-STRASBOURG F-67412 ILLKIRCH Cedex
> Tel: +33 3 68 85 24 34
>
--
Mark Taylor Astronomical Programmer Physics, Bristol University, UK
m.b.taylor at bristol.ac.uk http://www.star.bristol.ac.uk/~mbt/
More information about the semantics
mailing list