08 - Transfer Manager

Owned by Anastasia Alexov (Unlicensed)
Apr 29, 2022

Design Diagram

The Transfer Manager is a Globus Connect Server within the Data Center. More information on Globus Connect Serves can be found here: https://www.globus.org/globus-connect-server.

The purpose of this server is to present selected storage to the Globus services. The storage devices have their contents configured to be accessible on the Data Center endpoint. Users then use the Globus services to transfer data to an endpoint they are authorized for whether that be a University, Cloud or personal resource.



Key Concept: Globus Storage Connectors

Globus can connect to multiple different types of storage devices through connectors. In the data center, the Ceph connector and POSIX connector will be used. 

More information can be found here: https://docs.globus.org/globus-connect-server-v5-installation-guide/#introduction.

Key Concept: Globus Constraints

There are a few limits on using the file transfer service. Some of these limits are listed below:

  • Only 3 or less active transfer tasks

  • Only 100 or less pending transfer tasks

  • Only 100,000,000 files or less in a single transfer task

  • Only 5,000,000 files or less in a single directory

  • A single user can only own 1,000 endpoints or less. This total includes both host endpoints and shared endpoints owned by the user.

  • Only 100 effective ACLs per user on an endpoint are allowed

  • Only 1,000 ACLs per endpoint in total are allowed

  • There can only be 20 transfer API requests or less per second per authenticated user. Metering is done over 10 second periods to allow for bursts.

For more information, users can consult the Globus documentation found here: https://docs.globus.org/faq/transfer-sharing/#are_there_any_limits_on_using_the_file_transfer_service.