<div dir="ltr">Hi Markus, hi all,<div>if this reply is confusing it's probably me not completely getting the points.</div><div>So...if you get lost, discard this reply from your inbox.<br><div class="gmail_extra"><br><div class="gmail_quote">2015-04-23 11:57 GMT+02:00 Markus Demleitner <span dir="ltr"><<a href="mailto:msdemlei@ari.uni-heidelberg.de" target="_blank">msdemlei@ari.uni-heidelberg.de</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Dear DAL, dear DM,<br>
<br>
On Thu, Apr 23, 2015 at 09:51:46AM +0200, Marco Molinaro wrote:<br>
> regarding this topic I have a small use case that comes from a (currently<br>
> custom) set of services whose aim is to allow velocity spectra analysis of<br>
> galactic FITS cubes.<br>
<br>
That's a perfect use case for obscore+datalink, I'd say.<br></blockquote><div><br></div><div>sure, it can also fit a SIAv2 plus AccessData, I think.</div><div>Honestly I'm quite puzzled on the way to go, I'll vote for providing both, at least on the long term,</div><div>even if currently a related service is already a TAP one, so obscore should be a direct extension.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">> a - a super-set of FITS cubes from non-homogeneous galactic surveys and<br>
> pointed archives in the radio band is deployed to allow velocity spectra<br>
> analysis<br>
> b - the first step for the user is to search in this set for available data<br>
> along a line-of-sight, with possible filtering on a cone around it, or a<br>
> box around it, limiting the velocity range, selecting explicitly one/more<br>
> survey(s) by name, species, transition, ...<br>
<br>
It seems to me most of the necessary metadata already is in obscore,<br>
no?<br></blockquote><div><br></div><div>also this is right.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">> c - the search output (which includes something along the lines of a<br>
> PublisherDID) is then used to explicitly cut the needed cube(s) to make<br>
> data transfer affordable (in the near future merging of adjacent<br>
> "same-survey" cubes will be also implemented)<br>
<br>
And here I'd argue that's a Datalink thing. There's just too many<br>
sorts of processing one could do on data products to have any<br>
hopes of describing them in a single database table, and datalink<br>
lets you do exactly what you're asking for with minimal overhead on<br>
both the table and the client (it will typically have to request one<br>
small file per cutout, of course, but given the transfer volumes<br>
we're talking about here on the data side that's neglible).<br></blockquote><div><br></div><div>one small file for each cutout call is what current custom service does.</div><div>I'm not saying I will not use Datalink, but I'd like to see whether that's the only solution.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Conversely, just having the pixel sizes of the cube (as in the +6<br>
columns proposal) won't really help you for your use case either, and<br>
even if that information could, you'd still have to have some<br>
descriptor of the access service somewhere, and so you'd have to use<br>
datalink either way.<br>
<br>
> The need for WCS information in the output of the search comes from the<br>
> idea of allowing the client side to build correct cutout queries to the<br>
<br>
Well, let me do a general plea here: "Keep data discovery and data<br>
access separate as much as you can." Datalink is the model to<br>
efficiently perform that separation.<br></blockquote><div><br></div><div>again, current custom service does so, but uses the same parameters to do both,</div><div>it's just for logical reasons that the endpoints of the services will be detached, otherwise</div><div>simple parameter discrimination between the two would do (in my case).</div><div>You're probably right my use case is already covered from DAL point of view, maybe the only</div><div>thing not spoken about a lot is the velocity axis of the cubes.</div><div> </div><div>[cutting...] I think I follow you, but I'm not completely convinced...could be my use case is not so complex</div><div>in these terms.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I give you, though, that there are open issues from a practical<br>
perspective. Mine are:<br>
<br>
(1) certain types of queries (e.g., "give me all datasets that have a<br>
certain axis type in any position") aren't really too well suited<br>
for going through indices.<br>
<br>
(2) there might be major *discovery* use cases that require additional<br>
information on the axes<br>
<br>
On (1), I've already written something in<br>
<a href="http://mail.ivoa.net/pipermail/dm/2015-April/005150.html" target="_blank">http://mail.ivoa.net/pipermail/dm/2015-April/005150.html</a>, which I<br>
think hasn't been disputed yet.<br></blockquote><div><br></div><div>In my case it would be finding datasets having velocity axis</div><div>somewhere (usually it's the 3rd axis, but </div><div>in general it shouldn't change if it's in another position).</div><div>But I yet have to understand what to put to identify the velocity axis.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On (2), I can well imagine they exist, but I'd still hope we can avoid<br>
expanding obscore by 20% to satisfy those. Let's identify them if<br>
they're there, shall we?<br>
<br>
Let me mount the soapbox once more (I'm done in a few lines): Think<br>
of our adopters. Based on what I hear from DaCHS' users and even a<br>
sizable crowd on this list, I'm convinced that additional fields in<br>
DMs are being paid for in terms of takeup (not to mention that people<br>
tend to put junk in fields whose purpose they don't understand).<br>
<br>
For the sake of takeup, please don't add fields without a strong,<br>
validated use case that cannot be sanely satisfied in any other way.<br></blockquote><div><br></div><div>I find it difficult to reply this soapbox buildup, because I found myself many</div><div>times in the dilemma of using or not (also some SHOULD) fields due to lack</div><div>of description of what was required.</div><div><br></div><div>I understand (already said so in my previous mail) your concern on widening</div><div>the number of table fields. I agree we should have use cases, to be sure we </div><div>don't leave someone out and find ourselves in troubles with new revisions when </div><div>they require a major one for back-compatibility issues </div><div>(oh, god, breaking back-compatibility!).</div><div><br></div><div>I also think we need clear guidelines if we want takeup to increase when </div><div>resources are limited or otherwise spent.</div><div><br></div><div>ciao</div><div> Marco</div><div><br></div></div></div></div></div>