Transfer of large data

Masahiro TANAKA masahiro.tanaka at nao.ac.jp
Thu Jul 21 01:31:41 PDT 2005


Dear all,

Have you experienced large data transfer (e.g. >100MB) between
distributed VO machine?  Would you tell your practical solution?

Our JVO team has built a SkyNode server which receives a query and
returns a result VOTable through Web Services, using Apache Axis.
Then we encounter a performance problem in large data transfer
using SOAP messages; Large amount of memory is allocated to hold
the entire data as Java objects. It slows processing speed, too.
Sending large data through SOAP messages seems unrealistic.

So we are considering SkyNodes pass result VOTable using:
 1. Staging (Result data are separately transferred through HTTP or
	     FTP, like SIAP)
 2. Attachment
Both will need some extension to SkyNode protocol.

Masahiro Tanaka



More information about the dal mailing list