SAMP questions

[Mark Taylor]

Tom,

I'm CC'ing this response to the apps-samp list for information
and in case others want to chip in.

On Wed, 6 Sep 2017, Thomas Robitaille wrote:

> Hi Mark,
> 
> I've finally had a chance to develop a SAMP plugin for glue and it
> works nicely with TOPCAT:
> 
> https://youtu.be/YB1rMCxzA3E

Excellent!  Glue is a welcome addition to the list of SAMPified
applications.  I've added a note to the (quite incomplete) list
at http://wiki.ivoa.net/twiki/bin/view/IVOA/SampSoftware.
Nice video too.

> However, I have a few SAMP-related questions/comments based on the
> list of mtypes at:
> 
> http://wiki.ivoa.net/twiki/bin/view/IVOA/SampMTypes#table_load
> 
> - It would be nice to add a new mtype for loading 3 or
> higher-dimensional FITS files for viewers that support it. I guess the
> issue is that if we used image.load.fits for that, some viewers may
> truly only be able to handle the 2-dimensional case. An mtype for
> higher-dimensional data could include a parameter with the number of
> dimensions to avoid having to load the cube to find out?

It's a reasonable suggestion, and I seem to remember it coming
up in discussion before, at least as a possibility.
I think the reason it hasn't been done is because nobody has
felt the need to move cubes around between SAMP clients in
this way up till now.

As mentioned in the "Process" section of the MType wiki page
you reference above, if you think such an MType (cube.load.fits?)
would be useful and can get some other application developers
to agree to implement it, then go ahead, and add an entry
to the wiki page if it seems to be worth publicising.
The process is pretty informal, but obviously you won't get it
working in deployed clients on day 1 since multiple applications
(well, at least 2) have to implement and deploy the same thing
for it to be useful.

A less respectable alternative in this case might be just to try
using image.load.fits with a N-dimensional (N>2) FITS file.
I've just tried sending a 3d FITS cube to ds9 using image.load.fits,
and it does load it as a cube.  Given the stated semantics of
image.load.fits ("Loads a 2-dimensional FITS image.") that's obviously
abuse of the protocol, on the other hand it works ...

> - The table.select.rowList message can get very inefficient for tables
> with 1000s of rows where many are selected - I think part of the issue
> is that encoding a row number like 4544 as a string is pretty
> inefficient. Could we create a new message such as
> table.select.rowMask where we can transfer a boolean mask of selected
> rows, e.g. encoded as a single base64 string? Then applications can
> decide whether it's more efficient to send one or the other.

You're right, especially since SAMP infrastructure (hub and client
implementations) are supposed to be able to assume that message
bodies are not too large (in practice, that they can be stored in
memory without causing trouble).  Again, I guess this just hasn't
come up as a practical problem up till now.

If use of the existing table.select.rowList MType looks like it's
actually causing trouble, and if you want to come up with a
specification for this, I'd consider supporting it in topcat.

However, there is a downside to defining multiple MTypes that
do basically the same job: it increases the implementation burden 
on applications that want to receive this kind of message,
and/or complicates the logic required when sending such messages
(have to decide which variant is more efficient, and work out which
clients support which variant, and possibly send different variants
to different recipients).  In the end it is likely that
interoperability suffers.  These considerations are why defining
new MTypes along the lines of (e.g.) table.load.your-favourite-format
is generally discouraged.

> - How should applications react when receiving two table.load.votable
> messages with the same table-id and url? At the moment if I send two
> identical messages from glue, TOPCAT loads the table twice - see
> astropy code here to send the table:
> 
> https://gist.github.com/astrofrog/6e10d0375859055adb29780d03772c9d
> 
> This can happen if I broadcast a table, then load up two other
> applications and broadcast again. Should table-id not be enforced to
> be unique within a given application?

I hadn't thought about it.  That sounds reasonable, but I think you're
right at present topcat just loads a table as if it hadn't seen the
table-id before.  Probably it ought to return an error response instead.
I'll think about fixing that.

> - Should a table.select.rowList message be ignored if a table.load.*
> message hasn't been sent with matching table-id or url beforehand?

I'd say the receiving client should return an error response
(which is what topcat does).
 
> - Are there any plans to add mtypes for sending small datasets over
> SAMP directly rather than having to read/write vo tables, which can be
> a bit slow?

No there are not plans.  This suggestion has come up before,
and I admit it looks like a straightforward way to do things.
But because of the disadvantages of having multiple MTypes for
the same job, and the restriction on unlimited sized message bodies,
both mentioned above, I'm not very keen on the idea.  But if
others want to argue differently, feel free!

Mark

--
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor at bris.ac.uk +44-117-9288776  http://www.star.bris.ac.uk/~mbt/