Natural language queries with LLMs via MCP servers

[czwolf]

Hi Tim, all, 

During the summer 2025 I set up a public MCP server (implementing the Streamable http transport) to expose a SLAP2 data access service in a way that LLM could extract and manipulate data. 

It worked just fine, but was not usable: the result from the MCP response directly goes in the context window. Even flagship models have a context of 1Mb and any “real” query produced so many data that the context was immediately saturated. 

We adopted then the “skill” approach (this is what you describe as the markdown giving instructions to the LLM). Check the skill folder of https://github.com/VAMDC/pyVAMDC to see what we made. This approach works perfectly, and we are very happy with this, but it requires a sandbox so that the LLM can run system command and download files to parse. This is the case for Claude, ChatGPT and we also achieved this locally using open-wighted models + OpenWebUI with the OpenTerminal extension.

The very true force of LLM is that they can adapt even if the vocabulary of each data provider is slightly different. They can cope with the differences and align the sense - they do not need necessarily a fixed vocabulary. But this can help. 

I recently participated in writing a recommendation from the Research Data Alliance where we tried to define some “standard” for agentic tool for research - maybe this could be useful for the IVOA community too : https://www.rd-alliance.org/groups/data-director-agentic-ai-blueprint/outputs/data-director-agentic-ai-tool-blueprint/

All the best, 

Carlo.



> On 27 Jul 2026, at 17:09, Tim Jenness via interop <interop at ivoa.net> wrote:
> 
> At the recent SPIE meeting in Copenhagen, Ashley Barnes from ESO gave a talk on their experiments with teaching an LLM to query their data archive. They looked at making a dedicated agent taught explicitly but this cost them real money, and they looked at writing markdown instructions that the LLM could read to learn about the specifics of the archive center. This led me to ponder whether we should be talking about some kind of standardized MCP server interface (https://modelcontextprotocol.io/docs/getting-started/intro) that an LLM could query to work things out about the archive.
> 
> This seems to be the way that many services are heading (my photos application has an MCP server to let an LLM look for photos efficiently). On the one hand we have all these open protocols so that in theory an LLM can work it all out and form the right query for a TAP server by querying registry and looking at TAP_SCHEMA, and that should work. MCP might be able to let an agent do the same thing but using fewer tokens, even if the end point is the agent sending off a TAP query.
> 
> Is anyone else thinking about this? Can someone from ESO ask Ashley to join IVOA Slack to discuss this?
> 
> Each data center could put their own MCP server up with their own targeted API and that might be fine, but pooling our experience in terms of what helped and what made things worse would be really helpful and if we end up with an IVOA Note containing advice on natural language queries that would be great. If we ended up with a standardized vocabulary for MCP servers that might be even better.
> 
> -- 
> Tim Jenness
> Rubin Observatory

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/interop/attachments/20260728/3159a1c8/attachment.htm>