[p3t] P3T mtg - Mon, Jun 24 @20 UTC

[Russ Allbery]

Joshua Fraustro via p3t <p3t at ivoa.net> writes:

> We're currently using celery w/ redis at MAST for job queues/execution
> and have had good results, though I'm sure it doesn't fit your
> particular Rubin-specific quirks.

Oh, yes, good call, I should look at Celery again.  I really wanted arq to
work because we're using it very successfully for other non-IVOA things
and it's maintained by the same author as Pydantic, but this limitation is
fairly serious.  We were using Dramatiq before, which has the proper
backend support, but it doesn't have an async interface on the frontend.

Unfortunately, each time I change the queuing system, it's another week of
work just to get back to where I was.  :/  This probably means that I'm
missing some abstraction layer, but I keep thinking we'll land on one
queuing system and not have to change it again.

> I developed a package for MAST called vo-models
> [https://vo-models.readthedocs.io/latest/] that provides Pydantic models
> for IVOA service protocols with 1-to-1 mappings to their XML
> representations via pydantic-xml. It supports round-tripping from an XML
> payload->pydantic model->dict/json->XML. We currently use it for our
> TAP/UWS service. I've written models for UWS & VOSI
> (Availability/Tables) with drafts for VODataservice and VOResource on
> the way. Much like you've described, when implementing it into our
> services, we have to sub-class the UWS Parameters model with whatever we
> expect to be specific for that service.

Thank you for reminding me about this!  I had remembered that you were
using pydantic-xml and that was on my list to look at (I hadn't yet since
the XML code we already had seemed to be working), but had completely
forgotten that you also had a publicly-available library.

I will take a look at this as soon as I get back from vacation.  This
sounds really useful.

-- 
Russ Allbery (eagle at eyrie.org)             <https://www.eyrie.org/~eagle/>