Automatically Upload Files from a Windows Folder to Amazon S3

Getting Started#

Basic Concept#

Automatically upload files created in a Windows folder to Amazon S3

In business environments, various programs and systems create files in specific folders on Windows servers.

For example, business systems may generate daily reports, data-processing programs may save CSV files, and backup programs may create backup files in designated locations.

To use these files in the cloud, a person may need to check the files and upload them directly to S3 or run a separate upload program.

Automatic Windows folder upload is a method that directly connects the Windows environment where files are created to Amazon S3 and automatically uploads files matching specified conditions.

By connecting the folder where files are created to the S3 storage location and configuring which files to upload and when, you can automate the entire flow from file preparation to cloud storage.

This allows required files to be automatically applied to the cloud environment according to specified rules without repeatedly performing Check File → Manual Upload → Save to S3 after files are created in the Windows folder.

Upload Flow#

Automatically continue from file creation through S3 storage

Automatic upload can start when a file is created or changed in a specified Windows folder.

The Flow first checks files in the configured folder and determines whether they match conditions including filename, extension, and path.

Files matching the conditions are transferred to the specified Amazon S3 bucket and storage path.

After the transfer completes, you can check the actual upload results and processing status for each file through the Run.

For example, you can configure the following flow.

With this configuration, simply creating a file in the business system can automatically continue the subsequent cloud upload task.

Workflow Change#

Reduce repetitive manual uploads after file creation

When files created in Windows need to be stored in S3, the existing process may require checking whether files were created, selecting the required files, and manually running the upload.

If the file creation location and S3 storage location are fixed for recurring work, the same process must be performed repeatedly.

With automatic upload, you can connect the file creation location and cloud storage location in advance and automatically process tasks after file creation.

Before and After Upload

For example, if daily reports, data-processing results, or backup files must be stored in the same S3 location, you can reduce repetitive tasks from file creation through upload.

In this way, automatic Windows folder upload directly connects the system that creates files with the cloud environment that uses them.

IT Engineer#

Environment Connection#

Connect the Windows file path to the Amazon S3 storage location

To configure automatic upload, first connect the Windows device where files are created to Amazon S3.

In Source, specify the Windows device and folder path where files will be checked. In Target, configure the S3 bucket and storage location where files will be uploaded.

For example, assume files are created in the following Windows path.

D:\Data\Export

You can connect files created in that folder to upload to the following S3 location.

s3://company-data/export/

Configure the following items together in the transfer environment.

This allows you to clearly configure which Windows folder's files are stored in which S3 location.

You can also connect multiple folders on a single Windows server to different S3 paths.

Using this structure, files created on a single device can also be automatically separated into different S3 storage areas according to business purpose.

Detection Conditions#

Configure which files to upload and when

After connecting Windows and S3, configure the conditions that start automatic upload.

Instead of uploading every file, you can run only when specific files are created or limit upload targets based on specified extensions and filenames.

For example, you can use the following conditions.

For example, you can upload only CSV files from the /daily folder to S3 and store only ZIP files from the /backup folder in a separate S3 path.

With this configuration, even on a single Windows device you can create different upload flows according to file type and creation location.

Flow Configuration#

Connect file detection and S3 upload into one Flow

After configuring the Source, Target, and execution conditions, connect them into a single Flow.

The Flow checks files in the Windows folder, determines upload targets according to the configured conditions, and then transfers the files to Amazon S3.

Basically, one folder can be connected to one S3 path, but in actual business environments this can be expanded in several ways.

Collect Multiple Folders into One S3 Location#

Result files or log files created on multiple servers can be collected into a single S3 bucket.

Route One Folder to Multiple S3 Paths#

You can also configure data files and backup files to be stored in different S3 paths according to file type.

By connecting file conditions and target locations in a Flow, you can create an automation structure suited to the actual file-processing environment, not just a simple upload task.

Result Check#

Check automatic upload tasks and per-file S3 results

When the Flow runs, you can check the progress and results of automatic upload tasks in Runs.

In the execution list, you can check the status of multiple automatic upload tasks. When you select a specific Run, you can check the actual processed files and Source and Target information in detail.

In each Run, you can check the following information.

In Run details, you can check not only the overall task but also the processing status of individual files.

This allows you to check whether automatic upload ran correctly and manage the process of applying actual files to the specified S3 location.

Follow-up Connection#

Automatically continue to data processing and notifications after S3 upload

After files are uploaded to Amazon S3, you can automatically connect the next task based on the results.

For example, when data files are stored in S3, you can start an analytics task, or when a backup file upload completes, notify the responsible person of the result.

For example, after uploading daily CSV files to S3, you can start a data-processing Flow, or notify the operations manager when a backup file upload completes.

This connection goes beyond simply uploading files from a Windows folder to S3.

File Creation → Automatic Detection → Check Conditions → S3 Upload → Check Results → Follow-up Processing

This can be expanded into a single file automation flow.

Failure Handling#

Check upload status and rerun required files or tasks

If a situation requiring additional review occurs during automatic upload, use the Run details to identify the stage where processing did not proceed.

You can check the Windows Source connection status and original file path together with the Amazon S3 Target access status and storage location.

Failure handling can proceed through the following flow.

When checking the upload environment, review the following items together.

For example, if the Windows folder path changed or the S3 storage location cannot be accessed, adjust the relevant settings and rerun the task.

After rerunning, use the new Run to check whether the files were successfully applied to Amazon S3.

Through this process, you can reliably operate an automatic upload flow from the Windows file creation environment through Amazon S3 storage and follow-up work connections.

Developer#

Register automation in code to monitor a Windows folder and transfer files to S3

Prepare the following items before starting.

Python, Node.js define and use a minimal api() helper that directly calls REST (see API Call Recipe). Java, C# use the bundled-source InnorixClient and Json helper (J for C#), which include encode_path and now_iso. Replace the device identifiers (device-win-01, device-s3) and paths with actual values.

Create Monitoring Automation#

Create sync automation that monitors the source folder. Real-time monitoring is configured with transferType: "sync" and the synchronization options (syncType, watchFolderType) in transferOptions. For paths, use a token (encode_path) that combines the device identifier and encoded path.

def create_watch_upload(source, source_path, target, target_path,
                        action="overwrite", watch_type=1):
    body = {
        "name": "windows to s3 watch",
        "flowName": "windows to s3 watch",
        "transferType": "sync",
        "isUpcoming": False,
        "timezone": "Asia/Seoul",
        "details": [{
            "senderId": source,
            "receiverId": target,
            "sourceItem": [{
                "hash": encode_path(source, source_path),
                "filePath": str(source_path or "").replace("\\", "/"),
                "isDir": True,
            }],
            "targetPath": encode_path(target, target_path),
            "step": 1,
            "transferOptions": {
                "noSchedule": True,
                "target-action": action,
                "send-fileoption": {},
                "syncType": 1,                  # one-way sync
                "watchFolderType": watch_type,  # 1 = on create, 2 = on modify
            },
        }],
        "schedules": [{
            "type": "none", "startDateType": "now",
            "startDate": now_iso(), "timezone": "Asia/Seoul",
        }],
        "step": 1,
    }

    return api("POST", "/api/automations", body)["automationId"]


automation_id = create_watch_upload(
    "device-win-01", "C:/data/export",
    "device-s3", "my-bucket/incoming")

static String createWatchUpload(InnorixClient client, String source, String sourcePath,
        String target, String targetPath, String action, int watchType) {
    Map<String, Object> detail = Json.newObj(
            "senderId", source,
            "receiverId", target,
            "sourceItem", Json.newArr(Json.newObj(
                    "hash", InnorixClient.encodePath(source, sourcePath),
                    "filePath", sourcePath.replace("\\", "/"),
                    "isDir", true)),
            "targetPath", InnorixClient.encodePath(target, targetPath),
            "step", 1,
            "transferOptions", Json.newObj(
                    "noSchedule", true,
                    "target-action", action,
                    "send-fileoption", Json.newObj(),
                    "syncType", 1,                  // one-way sync
                    "watchFolderType", watchType)); // 1 = on create, 2 = on modify

    Map<String, Object> body = Json.newObj(
            "name", "windows to s3 watch",
            "flowName", "windows to s3 watch",
            "transferType", "sync",
            "isUpcoming", false,
            "timezone", "Asia/Seoul",
            "details", Json.newArr(detail),
            "schedules", Json.newArr(Json.newObj(
                    "type", "none", "startDateType", "now",
                    "startDate", InnorixClient.nowIso(), "timezone", "Asia/Seoul")),
            "step", 1);

    return Json.str(client.apiObj("POST", "/api/automations", body, null), "automationId");
}

async function createWatchUpload(source, sourcePath, target, targetPath,
                                 action = "overwrite", watchType = 1) {
  const body = {
    name: "windows to s3 watch",
    flowName: "windows to s3 watch",
    transferType: "sync",
    isUpcoming: false,
    timezone: "Asia/Seoul",
    details: [{
      senderId: source,
      receiverId: target,
      sourceItem: [{
        hash: encodePath(source, sourcePath),
        filePath: String(sourcePath ?? "").replace(/\\/g, "/"),
        isDir: true,
      }],
      targetPath: encodePath(target, targetPath),
      step: 1,
      transferOptions: {
        noSchedule: true,
        "target-action": action,
        "send-fileoption": {},
        syncType: 1,                 // one-way sync
        watchFolderType: watchType,  // 1 = on create, 2 = on modify
      },
    }],
    schedules: [{
      type: "none", startDateType: "now",
      startDate: nowIso(), timezone: "Asia/Seoul",
    }],
    step: 1,
  };

  return (await api("POST", "/api/automations", body)).automationId;
}

const automationId = await createWatchUpload(
  "device-win-01", "C:/data/export",
  "device-s3", "my-bucket/incoming");

static async Task<string> CreateWatchUploadAsync(InnorixClient client, string source, string sourcePath,
    string target, string targetPath, string action = "overwrite", int watchType = 1)
{
    var detail = new JsonObject
    {
        ["senderId"] = source,
        ["receiverId"] = target,
        ["sourceItem"] = new JsonArray
        {
            new JsonObject
            {
                ["hash"] = InnorixClient.EncodePath(source, sourcePath),
                ["filePath"] = sourcePath.Replace("\\", "/"),
                ["isDir"] = true,
            }
        },
        ["targetPath"] = InnorixClient.EncodePath(target, targetPath),
        ["step"] = 1,
        ["transferOptions"] = new JsonObject
        {
            ["noSchedule"] = true,
            ["target-action"] = action,
            ["send-fileoption"] = new JsonObject(),
            ["syncType"] = 1,                 // one-way sync
            ["watchFolderType"] = watchType,  // 1 = on create, 2 = on modify
        },
    };

    var body = new JsonObject
    {
        ["name"] = "windows to s3 watch",
        ["flowName"] = "windows to s3 watch",
        ["transferType"] = "sync",
        ["isUpcoming"] = false,
        ["timezone"] = "Asia/Seoul",
        ["details"] = new JsonArray { detail },
        ["schedules"] = new JsonArray
        {
            new JsonObject
            {
                ["type"] = "none", ["startDateType"] = "now",
                ["startDate"] = InnorixClient.NowIso(), ["timezone"] = "Asia/Seoul",
            }
        },
        ["step"] = 1,
    };

    return J.Str(await client.ApiObjAsync("POST", "/api/automations", body), "automationId");
}

Caution Sync automation reads the monitoring path from sourceItem[0].filePath. If only hash (token) is provided and filePath is omitted, the monitoring path is not specified.

Check Execution#

Execution history accumulates each time the monitoring automation runs. Query the status and monitorId for each execution. The newest execution appears first in the array.

runs = api("GET", f"/api/automations/{automation_id}/executions") or []

for run in runs:
    print(run["startTime"], run["status"], run["monitorId"])

List<Object> runs = Json.asArr(client.api("GET", "/api/automations/" + automationId + "/executions"));

for (Object run : runs) {
    Map<String, Object> r = Json.asObj(run);
    System.out.println(Json.str(r, "startTime") + " " + Json.intOrNull(r, "status") + " " + Json.str(r, "monitorId"));
}

const runs = (await api("GET", `/api/automations/${automationId}/executions`)) || [];

for (const run of runs) {
  console.log(run.startTime, run.status, run.monitorId);
}

JsonArray runs = J.AsArr(await client.ApiAsync("GET", 
quot;/api/automations/{automationId}/executions"));

foreach (JsonNode run in runs)
    Console.WriteLine(
quot;{J.Str(run, "startTime")} {J.IntOrNull(run, "status")} {J.Str(run, "monitorId")}");

Each execution's monitorId connects to file-level results (see API Call Recipe for transfer status and file queries).

Pause, Resume, Delete Monitoring#

To temporarily stop monitoring, pause it; to turn it back on, resume it. To end monitoring completely, delete the automation. To change the monitored folder or target path, delete the existing automation and create a new automation with the changed path.

api("POST", f"/api/automations/{automation_id}/pause", {"pause": True})   # pause
api("POST", f"/api/automations/{automation_id}/pause", {"pause": False})  # resume
api("DELETE", f"/api/automations/{automation_id}")                        # delete automation (permanent)

client.api("POST", "/api/automations/" + automationId + "/pause", Json.newObj("pause", true));   // pause
client.api("POST", "/api/automations/" + automationId + "/pause", Json.newObj("pause", false));  // resume
client.api("DELETE", "/api/automations/" + automationId);                                        // delete automation (permanent)

await api("POST", `/api/automations/${automationId}/pause`, { pause: true });   // pause
await api("POST", `/api/automations/${automationId}/pause`, { pause: false });  // resume
await api("DELETE", `/api/automations/${automationId}`);                        // delete automation (permanent)

await client.ApiAsync("POST", 
quot;/api/automations/{automationId}/pause", new JsonObject { ["pause"] = true });   // pause
await client.ApiAsync("POST", 
quot;/api/automations/{automationId}/pause", new JsonObject { ["pause"] = false });  // resume
await client.ApiAsync("DELETE", 
quot;/api/automations/{automationId}");                                            // delete automation (permanent)

Implementation Result#

Applying this recipe allows you to automatically upload files from a Windows folder to S3 through the following flow.

Files created in the Windows folder can be continuously uploaded to S3 automatically without human intervention.