Git-Linked File Transfer

Getting Started

Core Concepts

Transfer files and artifacts that are not managed by Git separately

Git manages source code and its change history, while files such as build artifacts, deployment packages, and large datasets can be managed through separate transfer flows.

By connecting files created or prepared after Git operations to the systems where they are needed, you can build a management flow suited to each file's characteristics and usage environment.

Git Repository
      │
      │ Code Change
      ▼
Build / Processing
      │
      │ Output Files
      ▼
File Transfer
      │
      ▼
Target Device

For example, after source code changes and a build completes, the resulting packages or files can be automatically transferred to test servers, deployment servers, or data-processing systems.

Integration Flow

Automatically connect Git operations to file transfer

Perform build or file-processing tasks based on Git operations, then start the file transfer when the required files are ready according to the configured conditions.

After the transfer completes, continue the next task on the destination systems, such as testing, deployment, or data processing.

Git Operation
   │
   ▼
Build · File Processing
   │
   ▼
Generate Result Files
   │
   ▼
Run Transfer Task
   │
   ▼
Apply to Destination Systems
   │
   ▼
Continue to Next Task

Separation Benefits

Manage source code and business files using their respective methods

Using Git together with file transfer connects source-code change management with the transfer and use of separate files within a single business workflow.

This lets you manage each file according to its purpose while automatically starting file transfer after Git operations when needed.

IT Engineers

Environment Connection

Connect Git operations with file transfer systems

First, connect Git operations and the systems used for file transfer in a single flow.

Connect Git operations, build or file-processing environments, and the destination systems where result files will be used to build the complete file flow.

┌──────────────┐
│     Git      │
└──────┬───────┘
       │
       ▼
┌──────────────┐
│ Build Server │
└──────┬───────┘
       │
       ▼
┌──────────────┐
│ File Transfer│
└──────┬───────┘
       │
       ▼
┌──────────────┐
│Target Device │
└──────────────┘

Connecting each system and task prepares the basic environment that carries files created after Git operations into the next transfer stage.

Transfer Rules

Define files, destinations, and execution conditions using a single standard

Specify the files to manage separately and the destination systems for transfer, then define when the transfer should start based on Git or follow-up tasks.

Specify the paths and types of files to transfer and connect the destination servers or systems. Then use conditions such as a code change, build completion, or result-file creation as the transfer trigger.

Git / Build Event
        │
        ▼
 Start Condition
        │
        ├── Source
        │      └── File / Path
        │
        └── Target
               └── Device
                      │
                      ▼
                 Transfer Run

This lets you manage which files are transferred to which systems after which operations as a single execution standard.

Automated Flow

Automatically transfer files to designated systems after Git operations

Complete the automated flow from Git operations through file transfer using the systems and transfer rules configured above.

When a Git operation occurs, the connected build or processing task runs, and the prepared files are transferred to the designated systems according to the configured conditions.

Git Push
   │
   ▼
Build
   │
   ▼
Output Ready
   │
   ├──────────────┐
   │              │
   ▼              ▼
Test Server   Deploy Server
   │              │
   └──────┬───────┘
          ▼
       Complete

When one result file is configured for multiple test or deployment environments, the file can be automatically connected from the Git operation through to every required business environment.

Result Management

Manage transfer status and reprocessing flows together

Review executed file transfer tasks in Runs and their execution details.

You can identify which transfer task was triggered by a Git operation and manage processing status and transferred file information for each destination system together.

Git Workflow
      │
      ▼
 Transfer Run
      │
 ┌────┼───────┐
 ▼    ▼       ▼
Files Status Progress
      │
      ▼
 Result Review
      │
 ┌────┴─────────────┐
 ▼                  ▼
Completed      Check Required
                    │
                    ▼
              Run Details
                    │
                    ▼
               Condition Check
                    │
                    ▼
                  Retry
                    │
                    ▼
                 Complete

The main items to review are as follows.

When an execution result requires additional review, use the Run details and Audit Log to check file readiness, transfer paths, and destination-system connectivity before rerunning the required task.

Developers

Send build artifacts to multiple deployment targets and reflect transfer results in the pipeline exit code

Integration Preparation

Prepare shared request code and CI credentials

import os
import requests

BASE_URL = os.getenv("INNORIX_BASE_URL", "https://app.innorix.com").rstrip("/")
TOKEN = os.environ["INNORIX_ACCESS_TOKEN"]
WORKSPACE_ID = os.getenv("INNORIX_WORKSPACE_ID")   # optional; falls back to the current workspace

STATUS_COMPLETE = 2
TERMINAL = {2, 4, 5, 9, 99}          # complete / error / cancelled / partial / failed
NOT_SUCCEEDED = {4, 5, 9, 99}


def api(method, path, body=None, params=None):
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {TOKEN}",
    }

    if WORKSPACE_ID:
        headers["x-workspace-id"] = WORKSPACE_ID

    response = requests.request(
        method, BASE_URL + path,
        headers=headers, json=body, params=params, timeout=30,
    )

    payload = response.json() if response.content else {}

    if not response.ok:
        raise RuntimeError(payload.get("message") or f"HTTP {response.status_code}")

    return payload.get("data")


def is_terminal(detail):
    return detail.get("isTerminal", detail.get("status") in TERMINAL)

Determine the transfer status using the values below. There are five terminal states, and the successful state is Complete (2).

When calling from CI, inject the token as a pipeline secret. Do not commit it to the repository.

# GitHub Actions example
env:
  INNORIX_BASE_URL: https://app.innorix.com
  INNORIX_ACCESS_TOKEN: ${{ secrets.INNORIX_ACCESS_TOKEN }}

Design Artifact Paths

Include the commit or tag in the path to retain deployment history

Record in the path which code produced the artifact. Keeping separate version folders leaves previous artifacts available and makes rollback easier when problems occur.

import os
import subprocess


def git_ref():
    try:
        sha = subprocess.check_output(
            ["git", "rev-parse", "--short", "HEAD"], text=True).strip()
    except (subprocess.CalledProcessError, FileNotFoundError):
        sha = "unknown"

    return os.getenv("GIT_TAG") or sha


def target_path(base, ref):
    return f"{base}/{ref}"

If everything is overwritten in a single path, there is no target to roll back to.

Send Artifacts

Send build result files to destination systems

When sending files, explicitly set isDir: false in sourceItem. sourcePaths treats every path as a folder, so supplying an artifact file can cause the server to scan it as a folder, resulting in slowdowns or timeouts.

def deploy_artifact(source, target, artifact_path, base):
    ref = git_ref()

    transfer = api("POST", "/api/transfers/manual", {
        "sourceDevice": source,
        "targetDevice": target,
        "targetPath": target_path(base, ref),
        "sourceItem": [{
            "path": artifact_path,
            "isDir": False,
            "isFolder": False,
            "fileSize": os.path.getsize(artifact_path),
        }],
        "sendAllFolder": False,
        "transferOptions": {"target-action": "overwrite"},
    })

    return transfer["monitorId"], target_path(base, ref)

When sending a build directory as a folder, use sourcePaths with sendAllFolder: True.

api("POST", "/api/transfers/manual", {
    "sourceDevice": source,
    "targetDevice": target,
    "targetPath": target_path(base, ref),
    "sourcePaths": ["/build/output"],
    "sendAllFolder": True,
    "transferOptions": {"target-action": "overwrite"},
})

Because the same reference is usually built and uploaded again, use overwrite. With numbering, copies accumulate every time you redeploy.

Deploy to Multiple Targets

Send one artifact to multiple environments

A single transfer handles one destination system. To send to both test and staging environments, create separate transfers.

TARGETS = [
    ("device-test-01", "/deploy/app"),
    ("device-stage-01", "/deploy/app"),
]

transfers = {
    target: deploy_artifact("device-build-01", target,
                            "/build/output/app.tar.gz", base)[0]
    for target, base in TARGETS
}

def wait(monitor_id, timeout=3600, interval=3):
    deadline = time.time() + timeout

    while time.time() < deadline:
        detail = api("GET", f"/api/transfers/{monitor_id}")

        if is_terminal(detail):
            return detail

        time.sleep(interval)

    raise TimeoutError(monitor_id)


def failed_files(monitor_id):
    result = api("GET", f"/api/transfers/{monitor_id}/files", params={
        "state": "any", "size": 500,
    }) or {}

    return [r for r in (result.get("children") or [])
            if r.get("status") in NOT_SUCCEEDED]


def retry_failed(monitor_id):
    rows = failed_files(monitor_id)

    if not rows:
        return 0

    api("POST", f"/api/transfers/{monitor_id}/retry", {
        "filesRetry": [
            {"filePath": r["sourceFilePath"], "isDir": bool(r.get("isFolder"))}
            for r in rows
        ]
    })

    return len(rows)

results = {}

for target, monitor_id in transfers.items():
    results[target] = wait(monitor_id, timeout=3600)

failed = [t for t, d in results.items() if d["status"] != STATUS_COMPLETE]

Even if one target fails, continue checking the others so you know how far the deployment has progressed. Stopping at the first failure makes it impossible to determine the redeployment scope.

Large Artifact Transfer

Control large-artifact throughput using speed and concurrency options

For large artifacts such as container images or datasets, use throughput options to control transfer speed.

api("POST", "/api/transfers/manual", {
    "sourceDevice": source,
    "targetDevice": target,
    "targetPath": target_path(base, ref),
    "sourcePaths": ["/build/output"],
    "sendAllFolder": True,
    "transferOptions": {
        "target-action": "overwrite",
        "networkLevel": 3,           # throughput priority level
        "concurrentTransfers": 8,    # concurrent transfers
    },
})

If the CI runner and deployment targets share the same network link, apply a rate limit so other tasks are not affected.

"transferOptions": {
  "target-action": "overwrite",
  "limitRate": 51200
}

The unit for limitRate is KB/s.

Pipeline Integration

Use the transfer result as the pipeline exit code

CI determines success or failure from the exit code. If the transfer fails, the pipeline must also fail.

import sys

if __name__ == "__main__":
    monitor_id, path = deploy_artifact(
        os.environ["BUILD_DEVICE"],
        os.environ["TARGET_DEVICE"],
        os.environ["ARTIFACT_PATH"],
        os.environ["DEPLOY_BASE"],
    )

    detail = wait(monitor_id)

    if detail["status"] != STATUS_COMPLETE:
        for row in failed_files(monitor_id)[:10]:
            print(row["sourceFilePath"], row.get("errorCode"), file=sys.stderr)

        print(f"retried {retry_failed(monitor_id)} files", file=sys.stderr)
        sys.exit(1)

    print(f"deployed: {path}")

Link Deployment History

Record monitorId together with commit information

Recording monitorId and commit information together links deployment history with code history.

record = {
    "monitorId": monitor_id,
    "commit": subprocess.check_output(
        ["git", "rev-parse", "HEAD"], text=True).strip(),
    "branch": os.getenv("GIT_BRANCH"),
    "buildNumber": os.getenv("BUILD_NUMBER"),
    "targetPath": path,
}

db.insert("deployments", record)

When a deployment issue occurs, find the history by monitorId and trace it back to the commit.

Review recent deployment history by time period.

from datetime import datetime, timedelta, timezone


def paginate(path, params=None, limit=200, max_pages=50):
    query = dict(params or {})
    query["limit"] = limit
    cursor = None

    for _ in range(max_pages):
        if cursor:
            query["cursor"] = cursor

        result = api("GET", path, params=query) or {}

        for record in result.get("data") or []:
            yield record

        pagination = result.get("pagination") or {}

        if not pagination.get("hasMore"):
            return

        cursor = pagination.get("nextCursor")

        if not cursor:
            return


end = datetime.now(timezone.utc)
fmt = "%Y-%m-%dT%H:%M:%SZ"

for row in paginate("/api/transfer-history", params={
    "startDate": (end - timedelta(days=7)).strftime(fmt),
    "endDate": end.strftime(fmt),
}):
    print(row["monitorId"], row.get("statusName"),
          row.get("targetDeviceName"), row.get("startDate"))