Summary
The DKIST Data Center Archive (DC) provides calibrated (a.k.a. Level 1, “L1”) data to the community via the User Portal for download. It does not provide access to raw (a.k.a. Level 0, “L0”) data. However, special requests can be made using a request form to the DC for raw data files. If the user has a justified reason to get these raw data, they can make a request to the DC for access. Requests will be reviewed on a case by case basis. It is assumed that a user will already have downloaded the L1 data before requesting the raw data. You are required to specify the dataset IDs from the L1 data that corresponds to the raw data you wish to download. Raw data is not searchable and cannot be filtered by instrument or other similar parameters. If the request is approved, the user will receive ALL the raw data for at least one observing program as executed at the telescope.
Note that if you are a PI who has been contacted by the data center regarding issues calibrating data from your proposal, do not use this form. Please respond to the DC using the email you were contacted by.
Warning: Raw data requests are for experts only. The dataset sizes are massive and working with these data requires a large amount of storage and processing power.
Warning: Most VBI data is speckle reconstructed at the telescope and this data is then downloaded to the data center. In such cases raw VBI data will not be available.
Submitting a raw data request does NOT guarantee that the request will be granted. Requests will be vetted on a case by case basis. Note that proprietary data can only be released to the PI and proposal team members.
Requests will be reviewed by a DKIST committee that will vet the request based on:
How does the request (if granted) benefit the solar community?
How does the request (if granted) benefit DKIST (e.g. donated improvement to calibration pipeline)?
Prerequisites to obtaining DKIST raw (L0) data
The user must have an active Globus account (see: How to create a globus account (for DKIST data))
The user must have an active endpoint for Globus data transfers (see: Globus Connect & Endpoints)
The user should already have downloaded and looked at the Level 1 datasets
Instructions
Visit the DKIST Data Portal Raw Data request form (link available soon)
Log into the Data Portal
Fill in the following mandatory fields on the request form; please be as descriptive as possible:
Datasets Requested: Dataset ID(s) where the FITS header keyword = “DSETID” in the L1 data
Justification: Statement for the reason for this raw data request
4. If the request is approved by the DKIST management, then
The user will receive a notification from the DC that the request was approved
Data Center staff will contact the user for additional information on requested datasets
Data Center staff will provide the raw data endpoint details for this Globus download as well as the allotted time the data will be made available to them
The user must initiate the Globus data transfer within the allotted time - the requested data will then be transferred to the user endpoint
If the user has requested notifications from Globus upon transfer completion, the user will receive a notification when the transfer completed (status: Success/Error)
5. If a raw data request is denied, the requester will be notified by email with the reason.
Note that the user will also need to download the DKIST pipeline software in order to process the data themselves, augmenting the parameters and processing procedures as they see fit. (See: <Pipeline Install/Run Instructions>)