File Transfer
Deliver the data AI Compute needs, exactly when it needs it.
Introducing INNORIX AI Data Delivery, which leaves your AI Infrastructure as-is and handles only the data movement between Storage and Compute.
File Transfer
Deliver the Data AI Compute Needs, When It Needs It.
AI Infrastructure already includes specialized systems that perform their own roles. Kubernetes and GPU Schedulers manage Compute Resources and Workload Placement, while ML Orchestrators such as Kubeflow manage the execution sequence and conditions of Training and Evaluation, as well as Artifact relationships. The problem AI Data Delivery is designed to solve is not to rebuild this Orchestration, but to move the actual files that make up Datasets, Models, Checkpoints, and Results between Storage and Compute at the required time.
- Object Storage
- Data Center
- File Storage
- Research Data
- AI Data Delivery
- GPU / AI Compute
- Checkpoint / Result
- Storage
INNORIX AI Data Delivery is a Data Delivery Layer that preserves the Scheduling and Pipelines of the existing AI Platform while applying High-Speed, Large File, High-Volume, and Recovery Capabilities to the Data Path of AI Infrastructure.
File Transfer
Keep the Existing AI Infrastructure and Handle the Actual Data Movement.
Where Compute is placed in an AI Pipeline, which Job runs first, and in what order Training and Evaluation execute remain the responsibility of the existing Scheduler and Orchestrator. The ML Platform can continue managing the URI, Metadata, and Dependency of Artifacts such as Datasets and Models.
INNORIX does not compete with these roles. It delivers the actual data between the Source and Target selected by the Orchestrator or existing system. It sends requested Datasets and Models to Compute, recovers interrupted Transfers, delivers Checkpoints and Results back to Storage or the next system, and then returns Status and completion Results. In other words, the existing AI Infrastructure decides Compute and Workloads, while INNORIX handles the actual File Movement required by those decisions.
AI Infrastructure Layer, What It Handles
- Scheduler / Orchestrator
- Select Compute
- Define Workload
- Request Data Delivery
- INNORIX
- Storage
- AI Compute
- Transfer Result
File Transfer
Connect the Data Path Between Storage and AI Compute Through One Transfer Layer.
Having AI Compute ready and having the Dataset required by that Compute ready are not the same problem. A Dataset may reside in Object Storage while Training runs on an On-Prem GPU Cluster. Data in internal enterprise Storage may need to be delivered to Cloud GPUs, or Results generated in the Cloud may need to be returned to Private Storage.
As AI Infrastructure expands across Multi-Cloud, On-Prem, Object Storage, and multiple Regions, adding Download, Temporary Staging, and Copy Scripts for every combination also multiplies Data Movement methods. Where possible, INNORIX configures the Source and Target as a direct Transfer relationship while managing Control through the Platform, allowing the Data Path between where the data currently resides and where the actual Compute runs to operate as a reusable Transfer Layer.
- Object Storage
- Data Center
- Private Storage
- INNORIX
- On-Prem GPU
- Cloud GPU
- AI Cluster
- Temporary Compute
- Regional Compute
- Edge Compute
- Result / Checkpoint
- Storage
File Transfer
Use the Same Data Delivery for Datasets, Models, Checkpoints, and Results.
In AI Workloads, data does not move only once before Training begins. Datasets are delivered to Compute, Checkpoints generated during Training are sent to Persistent Storage, completed Models are moved to Evaluation or Inference environments, and final Results are delivered back to Storage or the next system.
INNORIX does not redefine the meaning or Lineage of Datasets or Models. It handles the movement of the actual files that make up each Artifact. Pipeline stages and Dependencies remain under the existing AI Platform, while Dataset, Model Weight, Checkpoint, Evaluation Data, and Result movements that repeat across Storage → Compute → Storage are connected through the same Transfer Layer.
Data Type, Transfer Direction
- Dataset
- Training Compute
- Checkpoint Storage
- Model
- Evaluation Compute
- Model Storage
- Inference Compute
- Result
File Transfer
Handle TB-Scale Datasets and Massive Numbers of Files Together in the AI Data Path.
AI Dataset Transfer Profiles are not uniform. A workload may involve one enormous Model or a TB-scale Dataset, millions of Images and Samples, large Videos moving together with massive numbers of Metadata Files, or continuously added new data.
Rather than creating a separate Copy method specifically for AI, INNORIX applies its existing High-Speed, Large File, High-Volume, and Automated Transfer Capabilities to the Data Path of AI Infrastructure. This allows Transfer Capabilities appropriate to the size and number of files to be used within the same AI Data Delivery, from a single Large Artifact to massive numbers of Small Files, Mixed Datasets, and repeatedly generated Checkpoints.
Data Pattern, Required Transfer Type
- Large Model
- TB Dataset
- Millions of Images
- Video + Metadata
- Repeated Checkpoints
- AI Data Delivery
- Compute
File Transfer
Connect Transfer Performance and Recovery to Data Ready Time.
In AI environments, the instantaneous Transfer speed number is not the only thing that matters. Actual Data Readiness is affected by when the Dataset required by Compute becomes available, how quickly the necessary Model and Data can be delivered after new Compute is created, and from what point a long-running Transfer can continue after interruption.
INNORIX uses Parallel Transfer, Dynamic Concurrency, and Long-Distance Transfer to utilize the Capacity available on the Data Path, and Resume, Selective Retry, and Partial Recovery to continue interrupted Delivery. Rather than separating Performance and Recovery into independent features, it manages the entire process of preparing the Data required for AI Compute to actually start its Workload as one Transfer Operation.
- Compute Ready
- Dataset Delivery
- High-Speed Transfer
- Resume on Interruption
- Selective Retry on Failure
- Data Ready
- Workload Start
File Transfer
Let the Existing AI Orchestrator Request Data Delivery When It Is Needed.
AI Data Delivery is not a new AI Workflow Engine. The existing Orchestrator continues to manage Pipeline Components, execution sequence and conditions, Retry, Resource Requests, and Artifact relationships, while INNORIX connects as an independent step that executes the required Data Transfer within that Workflow.
After the Orchestrator determines the Workload and Target Compute and requests Data Delivery through an API, INNORIX executes the Transfer between the Source and Target and returns Status and Results. The existing Orchestrator decides which Job to run, when Training begins, and which Component executes after success, allowing the Data Path to be extended without redefining the AI Pipeline already in place.
Interface, Role
- Existing AI Orchestrator
- Prepare Workload
- Request Data Delivery
- Start Training
- Evaluate
- Continue Pipeline
- Request Data Delivery
- INNORIX
- Transfer Data
- Transfer Complete
File Transfer
Use the Same Structure from One-Time Migration to Recurring AI Data Delivery.
AI Data Movement may begin as a One-Time Migration that moves an existing Dataset or Model Repository into a new environment, and later expand into Scheduled Delivery of daily generated Datasets or Continuous Delivery of Artifacts required by Training, Evaluation, and Inference.
INNORIX connects the same Data Delivery through different execution methods using On Demand, Schedule, Event, and API. Because Orchestration remains with the existing AI Platform and only the actual File Movement is separated into the Transfer Layer, One-Time Migration and continuous AI Pipeline Data Movement can expand within the same operating model without rebuilding separate transfer architectures.
Delivery, Application
- One-Time Migration
- Scheduled Delivery
- Event-Driven Delivery
- Continuous AI Data Movement
File Transfer
Apply the Same Data Delivery to the Selected Endpoint Even as Compute Changes Dynamically.
AI Compute may not be a single fixed Server, but a Resource that is created or selected according to the Workload. In this case, the Scheduler and Orchestrator determine Compute Provisioning, Workload Placement, and the actual Target, while INNORIX delivers the required Dataset and Model to that Compute after the decision has been made.
This also clarifies the roles of AI Data Delivery and Dynamic Endpoint Transfer. AI Data Delivery focuses on which Dataset, Model, Checkpoint, and Result should be delivered, while Dynamic Endpoint Transfer extends the scope by connecting the Endpoint selected within changing Infrastructure as the actual Transfer Target. INNORIX does not intervene in Scheduling; it executes the decisions of the existing Infrastructure as Data Movement.
- Scheduler / Orchestrator
- Target Compute Selected
- Dynamic Endpoint
- INNORIX AI Data Delivery
- Dataset / Model Ready
File Transfer
Continue Data Delivery After Interruptions and Return Results to the Existing AI Platform.
For large-scale Datasets, how a failure is handled afterward is often more important than the failure itself. If only some files out of millions fail, or if a problem occurs after most of a TB-scale Dataset has already been delivered, preserving the existing Transfer State and continuing only the required scope reduces actual completion time compared with retransmitting the entire Dataset from the beginning.
INNORIX recovers Delivery through Resume, Selective Retry, and File-Level Results, while Runs, File Status, Transfer History, and Receipt provide visibility into the result of the Data Movement itself. Instead of inferring outcomes from Transfer Logs, the existing AI Platform can receive completion Results through APIs and Callbacks and continue to the next Pipeline stage.
- 10,000,000 Files
- Completed
- Completed
- Selective Retry on Failure
- Completed
- Completed
- Transfer Result
- Callback / Receipt
- Existing AI Platform
File Transfer
Keep AI Infrastructure Intact and Connect Data Delivery Through One Operating Model.
The scope of AI Data Delivery is not limited to a single Dataset Copy before Training. The same Transfer Capability can be applied across the complete Data Movement required to deliver data from Object Storage, Data Centers, Cloud, and Private Storage to the required Compute, and then move Models, Checkpoints, and Results generated by Training and Inference to their next destinations.
The existing AI Platform continues to manage GPU Provisioning, Resource Scheduling, Training Jobs, Pipeline Dependencies, Artifact Metadata, and Model Lifecycle. INNORIX handles Large and High-Volume Transfer, Recovery, Result Collection, and Monitoring in between, allowing the actual Data Delivery across Storage ↔ Compute ↔ Storage to be operated in one consistent way without changing the roles of the existing AI Infrastructure.
Existing AI Infrastructure, INNORIX
- AI Infrastructure, Provisioning, Scheduling, Pipeline, Training
- AI Data Delivery
- Storage, Dataset
- Compute, Workload
- Storage, Result
File Transfer
Configure Data Delivery All the Way to AI Compute.
Deliver Datasets stored in Object Storage, Data Centers, Cloud, and Private Storage to the required AI Compute, and connect Models, Checkpoints, and Results generated by Training and Inference to their next destinations.
Keep the existing Scheduler and AI Orchestrator in place while operating the actual Data Movement across Storage ↔ Compute ↔ Storage through a single Transfer Layer.