From m.b.taylor at bristol.ac.uk Fri Sep 8 16:23:34 2017 From: m.b.taylor at bristol.ac.uk (Mark Taylor) Date: Fri, 8 Sep 2017 15:23:34 +0100 (BST) Subject: SAMP questions In-Reply-To: References: Message-ID: Tom, I'm CC'ing this response to the apps-samp list for information and in case others want to chip in. On Wed, 6 Sep 2017, Thomas Robitaille wrote: > Hi Mark, > > I've finally had a chance to develop a SAMP plugin for glue and it > works nicely with TOPCAT: > > https://youtu.be/YB1rMCxzA3E Excellent! Glue is a welcome addition to the list of SAMPified applications. I've added a note to the (quite incomplete) list at http://wiki.ivoa.net/twiki/bin/view/IVOA/SampSoftware. Nice video too. > However, I have a few SAMP-related questions/comments based on the > list of mtypes at: > > http://wiki.ivoa.net/twiki/bin/view/IVOA/SampMTypes#table_load > > - It would be nice to add a new mtype for loading 3 or > higher-dimensional FITS files for viewers that support it. I guess the > issue is that if we used image.load.fits for that, some viewers may > truly only be able to handle the 2-dimensional case. An mtype for > higher-dimensional data could include a parameter with the number of > dimensions to avoid having to load the cube to find out? It's a reasonable suggestion, and I seem to remember it coming up in discussion before, at least as a possibility. I think the reason it hasn't been done is because nobody has felt the need to move cubes around between SAMP clients in this way up till now. As mentioned in the "Process" section of the MType wiki page you reference above, if you think such an MType (cube.load.fits?) would be useful and can get some other application developers to agree to implement it, then go ahead, and add an entry to the wiki page if it seems to be worth publicising. The process is pretty informal, but obviously you won't get it working in deployed clients on day 1 since multiple applications (well, at least 2) have to implement and deploy the same thing for it to be useful. A less respectable alternative in this case might be just to try using image.load.fits with a N-dimensional (N>2) FITS file. I've just tried sending a 3d FITS cube to ds9 using image.load.fits, and it does load it as a cube. Given the stated semantics of image.load.fits ("Loads a 2-dimensional FITS image.") that's obviously abuse of the protocol, on the other hand it works ... > - The table.select.rowList message can get very inefficient for tables > with 1000s of rows where many are selected - I think part of the issue > is that encoding a row number like 4544 as a string is pretty > inefficient. Could we create a new message such as > table.select.rowMask where we can transfer a boolean mask of selected > rows, e.g. encoded as a single base64 string? Then applications can > decide whether it's more efficient to send one or the other. You're right, especially since SAMP infrastructure (hub and client implementations) are supposed to be able to assume that message bodies are not too large (in practice, that they can be stored in memory without causing trouble). Again, I guess this just hasn't come up as a practical problem up till now. If use of the existing table.select.rowList MType looks like it's actually causing trouble, and if you want to come up with a specification for this, I'd consider supporting it in topcat. However, there is a downside to defining multiple MTypes that do basically the same job: it increases the implementation burden on applications that want to receive this kind of message, and/or complicates the logic required when sending such messages (have to decide which variant is more efficient, and work out which clients support which variant, and possibly send different variants to different recipients). In the end it is likely that interoperability suffers. These considerations are why defining new MTypes along the lines of (e.g.) table.load.your-favourite-format is generally discouraged. > - How should applications react when receiving two table.load.votable > messages with the same table-id and url? At the moment if I send two > identical messages from glue, TOPCAT loads the table twice - see > astropy code here to send the table: > > https://gist.github.com/astrofrog/6e10d0375859055adb29780d03772c9d > > This can happen if I broadcast a table, then load up two other > applications and broadcast again. Should table-id not be enforced to > be unique within a given application? I hadn't thought about it. That sounds reasonable, but I think you're right at present topcat just loads a table as if it hadn't seen the table-id before. Probably it ought to return an error response instead. I'll think about fixing that. > - Should a table.select.rowList message be ignored if a table.load.* > message hasn't been sent with matching table-id or url beforehand? I'd say the receiving client should return an error response (which is what topcat does). > - Are there any plans to add mtypes for sending small datasets over > SAMP directly rather than having to read/write vo tables, which can be > a bit slow? No there are not plans. This suggestion has come up before, and I admit it looks like a straightforward way to do things. But because of the disadvantages of having multiple MTypes for the same job, and the restriction on unlimited sized message bodies, both mentioned above, I'm not very keen on the idea. But if others want to argue differently, feel free! Mark -- Mark Taylor Astronomical Programmer Physics, Bristol University, UK m.b.taylor at bris.ac.uk +44-117-9288776 http://www.star.bris.ac.uk/~mbt/