MAXREC in Datalink

[alberto micol]

At ESO we have some rare cases whereby the list of progenitors of a dataset could be long various thousand entries.

In this case the datalink response is obviously huge, making an impossible user experience, and having clients hanging.
 
To solve this, we have done the following:

if the number of progenitors is below a modest amount (say 25) then the full provenance is shown

otherwise, no provenance is shown other than one record which points to the full provenance (still a valid datalink document).

Such record looks like this:

   <TR>
     <TD>ivo://eso.org/ID?ADP.2016-12-01T16:07:25.587</TD>
     <TD><![CDATA[http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2016-12-01T16:07:25.587&EXCLUDE=ALL&INCLUDE=progenitor]]></TD>
     <TD />
     <TD />
     <TD>#progenitor</TD>
     <TD>Link that brings to the actual list of progenitors (processing provenance)</TD>
     <TD>application/x-votable+xml</TD>
     <TD />
     <TD />
     <TD />
   </TR>
  
For that, we had to implement two ad-hoc input parameters, EXCLUDE and INCLUDE,
so that we can have full control of what is shown when.

With this mechanism, our clients (e.g. the ESO science portal) can query for anything they like, e.g.
to get only derivation and provenance, one would use: EXCLUDE=ALL&INCLUDE=progenitor&INCLUDE=derivation
or to get everything excluding provenance: INCLUDE=ALL&EXCLUDE=progenitor

The presence of INCLUDE or EXCLUDE removes the default behaviour of showing only 1 link-record if more than 25 entries exist.

I hope the above is still DataLink compliant… comments are welcome!

PS: We are about to release this new version of datalink possibly next week, along with all new versions of many archive services.
So, please do not try now the link above.

Alberto





 



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ivoa.net/pipermail/dal/attachments/20200330/a909e5a4/attachment.html>