[AI-interop] Natural language queries with LLMs via MCP servers

[Molinaro, Marco]

Dear all,

considering the amount of interest shown on this topic
at the interop and on the list (as well as slack, for those
using that tool too), a mailing list has been created to

- avoid mixing up with the interop general list
- continue this discussion and see where it might bring us

The mailing list, already in place is ai-interop, subscriptions
open at

http://mail.ivoa.net/mailman/listinfo/ai-interop

(as for most IVOA mailing lists, list membership is needed
to post to it)

Looking forward to the follow up!

Kind Regards
    Marco

PS - this message is CC'd to the ai-interop list to keep
initial information archived also there

Il giorno ven 31 lug 2026 alle ore 17:11 Sebastien Derriere via interop <
interop at ivoa.net> ha scritto:

> Hi all,
>
> As André said, we have been experimenting with LLMs in order to query CDS
> services in natural languages.
> The two main limitations at present (in my opinion) are the following :
>
> 1. LLMs want to please the user, and therefore will rather hallucinate some
> plausible answer, rather than admitting they don't know how to answer. This
> is quite frustrating. You can try to play with parameters and reduce
> temperature,
> but I've seen too many behaviours where you can not trust the answers.
> Examples include :
> - LLM invents some VizieR catalogue identifier, with an author and
> bibcode, reference
> paper in A&A, with table and column names, to "retrieve" information. All
> made-up.
> - Translation from natural language to TAP query provides some ADQL and
> astroquery
> code. Everything looks nice, except some table names and columns are
> complete
> inventions, and do not exist in the TAP schema.
>
> 2. Turning a prototype into an operational service requires serious
> computing
> resources, or a potentially very high cost in tokens used if you rely on a
> third-party
> infrastructure.
>
> So far, we have had some interesting results with MCP. I'd like to try
> testing skills
> in the near future to see if this can help improve reliability and
> performance.
> There are already collections of astronomy-related skills in this list of
> resources :
> https://www.skills.sh/
> (including some for ESO services : https://github.com/szampier/skills )
>
> Sébastien
>
> Le 29/07/2026 à 13:00, Schaaff Andre via interop a écrit :
>
> Hello Tim and all
>
> At CDS we (Thomas Boch, Sébastien Derrière and myself) worked on a MCP
> server implementation and configuration in the frame of the R&D around
> chatbots and AI for astronomical services and tools.
>
>  Even if it was continued  since Görlitz, Thomas had a talk in the joint
> DCP/KD Session in November 2025 to give an overview of the ongoing effort
> in this frame
>
> https://wiki.ivoa.net/twiki/bin/view/IVOA/InterOpNov2025DCPKD
>
> we will be happy to join discussions and  other actions
>
> regards
>
> André
> Le 27/07/2026 à 17:09, Tim Jenness via interop a écrit :
>
> At the recent SPIE meeting in Copenhagen, Ashley Barnes from ESO gave a
> talk on their experiments with teaching an LLM to query their data archive.
> They looked at making a dedicated agent taught explicitly but this cost
> them real money, and they looked at writing markdown instructions that the
> LLM could read to learn about the specifics of the archive center. This led
> me to ponder whether we should be talking about some kind of standardized
> MCP server interface (
> https://modelcontextprotocol.io/docs/getting-started/intro) that an LLM
> could query to work things out about the archive.
>
> This seems to be the way that many services are heading (my photos
> application has an MCP server to let an LLM look for photos efficiently).
> On the one hand we have all these open protocols so that in theory an LLM
> can work it all out and form the right query for a TAP server by querying
> registry and looking at TAP_SCHEMA, and that should work. MCP *might* be
> able to let an agent do the same thing but using fewer tokens, even if the
> end point is the agent sending off a TAP query.
>
> Is anyone else thinking about this? Can someone from ESO ask Ashley to
> join IVOA Slack to discuss this?
>
> Each data center could put their own MCP server up with their own targeted
> API and that might be fine, but pooling our experience in terms of what
> helped and what made things worse would be really helpful and if we end up
> with an IVOA Note containing advice on natural language queries that would
> be great. If we ended up with a standardized vocabulary for MCP servers
> that might be even better.
>
> --
> Tim Jenness
> Rubin Observatory
>
> --
> Andre Schaaff
> Responsable R&D du CDS
> Observatoire Astronomique de Strasbourg (ObAS)
> UMR 7550 Universite de Strasbourg - CNRS
> 11, rue de l'Universite
> F-67000 Strasbourg
>
>
> --
>
>    (((    Sebastien Derriere     sebastien.derriere at astro.unistra.fr
>   (. .)   Observatoire de Strasbourg       Phone +33 (0) 368 852 444
>  (( v ))  11, rue de l'universite        Telefax +33 (0) 368 852 432
> ---m-m--- F-67000 Strasbourg  France
>
>

-- 
Marco Molinaro
INAF - Istituto Nazionale di AstroFisica
Osservatorio Astronomico di Trieste
email marco.molinaro at inaf.it
tel. [+39] 333 33 20 564 [also Telegram]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/ai-interop/attachments/20260803/673a274a/attachment.htm>