A company’s data analytics work starts at 8 a.m. every day.Five systems prepare the data it needs, but once again, each one is ready at a different time.
Scenario
Through a fictional company whose data is ready at a different time every day, we look at the run conditions between jobs and how data moves.
08:00MONDAY MORNING
Today’s data readiness
Transaction systemReady07:12
Customer systemReady07:38
Service operations systemReady07:51
External data systemTransferringTBD
Data processing systemProcessingTBD
Scheduled analytics start
08:00
Data ready
3 / 5
Actual work status
Waiting for data
07:00Data created
08:00Analytics scheduled
09:00Downstream work
12:00Results in use
It’s time to start the analytics work. But data from two systems still isn’t ready.
Scene previewIt’s time to start work, but the data isn’t ready.
The gap between the work schedule and when data is actually ready.
From Monday morning to Tuesday morning, we follow the company’s data and work.
INNORIX in Practice
Monday morning07:00
Monday 07:00
Five systems start their day.
Every morning, the company uses the data created the day before for customer service, operations analysis, business analytics and more.
To support this, five separate systems each produce their own data.
MONDAY, 07:00
Data needed this morning
Source system
Today’s data
Expected ready
Expected size
Transaction system
Transaction records
07:15
84 GB
Customer system
Customer changes
07:30
12 GB
Service operations system
Service usage logs
07:45
264 GB
External data system
Inbound external data
07:50
48 GB
Data processing system
Processed business data
07:55
186 GB
Today’s schedule
08:00Update customer service data
08:00Analyze service operations
08:00Integrated analytics on all data
10:00Use analytics results
12:00Wrap up morning work
If all goes to plan, every dataset is ready before 8 a.m.
Live contextNow
07:00Monday morning
D
Data operations team07:00
We have started preparing the data for this morning’s analytics.
Data readiness0 / 5
Transactions0%
Customer data0%
Service ops0%
External data0%
Processed results0%
Downstream jobs now
Customer serviceWaiting
Operations statusWaiting
Integrated analyticsWaiting
Monday 07:51
Data for some of the work is ready.
Transaction data was ready at 7:12 a.m. Customer data and service operations data followed one after another.
But not every job at the company needs all five datasets.
DATA REQUIREMENTS
Each job needs different data.
Required data
Customer service
Operations status
Integrated analytics
TransactionsReady
Ready
—
Ready
Customer dataReady
Ready
—
Ready
Service ops dataReady
—
Ready
Ready
External dataPreparing
—
—
Waiting
Processed resultsPreparing
—
—
Waiting
Start condition
2 / 2
1 / 1
3 / 5
Customer service job
Ready to run
2 of 2 required data ready
Operations status job
Ready to run
1 of 1 required data ready
Integrated analytics job
Waiting
2 required data not ready
The data for integrated analytics isn’t ready yet,but other jobs can already start.
Live contextNow
07:51Monday morning
S
Service operations team07:51
All data for customer service is ready.
Data readiness3 / 5
TransactionsDone
Customer dataDone
Service opsDone
External data48%
Processed results32%
Downstream jobs now
Customer serviceReady to run
Operations statusReady to run
Integrated analyticsWaiting
Monday 08:00
It’s the scheduled start time.
It’s 8 a.m.
The external data system is still transferring over the network, and the data processing system is handling more data than usual.
The company’s integrated analytics needs all five datasets.
08:00
MONDAY MORNING
07:0007:3008:0008:3009:00
Transactions
07:12
Customer data
07:38
Service ops
07:51
External data
Processed results
Scheduled analytics start
Data ready
3 / 5
Scheduled start
08:00
Integrated analytics delivery
Waiting
A
Analytics operations team
Some data still isn’t ready. Integrated analytics is on hold.
Reaching the start time and being able to start are two different conditions.
Live contextNow
08:00Monday morning
A
Analytics operations team08:00
External data and processed results are not ready yet. Integrated analytics is on hold.
Data readiness3 / 5
TransactionsDone
Customer dataDone
Service opsDone
External data68%
Processed results42%
Downstream jobs now
Customer serviceCompleted
Operations statusIn progress
Integrated analyticsWaiting
Monday 08:24
Should every job wait for the last dataset?
External data has arrived. But the last dataset integrated analytics needs is still being processed.
Meanwhile, the customer service and service operations data has already been delivered to their own environments.
MONDAY, 08:24
The three jobs are in different states.
Customer service data delivery
Completed
07:38 — 07:42
Required data
TransactionsCompleted
Customer dataCompleted
Destination
Customer service server
2 / 2required data ready
Operations data delivery
Completed
07:51 — 08:13
Required data
Service ops dataCompleted
Destination
Operations analytics server
1 / 1required data ready
Integrated analytics data delivery
Waiting
Waiting for run conditions
Required data
TransactionsCompleted
Customer dataCompleted
Service ops dataCompleted
External dataCompleted
Processed resultsWaiting
Destination
Integrated analytics environment
4 / 5required data ready
When one job is delayed,unrelated jobs don’t have to wait with it.
Live contextNow
08:24Monday morning
C
Customer service team08:24
The customer service files have been delivered. We’ll check the integrated analytics results later.
Data readiness4 / 5
TransactionsDone
Customer dataDone
Service opsDone
External dataDone
Processed results76%
Downstream jobs now
Customer serviceCompleted
Operations statusCompleted
Integrated analyticsWaiting
Monday 08:47
The last required data is ready.
The data processing system has produced its last result file.
But a file existing doesn’t mean the data is fully ready. The job writing it may still be running, or only some of the related files may have been created.
So the company has defined, for each system, how to confirm that its data is ready.
DATA READINESS
When a file appears vs. when the data is ready
08:43
Writing files
/processed/2026-11-16/
result_001.parquetCompleted
result_002.parquetCompleted
result_003.parquetWriting
Next transfer waiting
08:47
Data ready
/processed/2026-11-16/
result_001.parquetCompleted
result_002.parquetCompleted
result_003.parquetCompleted
_SUCCESSDone marker
Next transfer running
Daily integrated analytics
INNORIX Flow
Data conditions required to run
Transactions deliveredMet
Customer data deliveredMet
Service ops data deliveredMet
External data deliveredMet
Processed results readyMet
Run conditionAll required conditions met
Delivery to the analytics environment started
Only after confirming the data the work needs is readydoes the next data movement begin.
Live contextNow
08:47Monday morning
D
Data operations team08:47
All required data is ready. Sending it to the analytics environment.
Data readiness5 / 5
TransactionsDone
Customer dataDone
Service opsDone
External dataDone
Processed resultsDone
Downstream jobs now
Customer serviceCompleted
Operations statusCompleted
Integrated analyticsTransferring
Monday 09:06
Analytics results aren’t ready all at once either.
At 8:49 a.m., the required data reached the integrated analytics environment, and the existing analytics system began processing it.
This system, too, produces different kinds of results at different times.
MONDAY, 09:06
Three jobs waiting on analytics results
The company’s existing analytics system
Daily analytics processing
Per-customer resultsGenerated
Per-service resultsGenerating72%
Full integrated resultsGenerating38%
Per-customer results
09:06
Transfer started
Customer service environment
Per-service results
Preparing
Waiting
Service management environment
Full integrated results
Preparing
Waiting
Business analytics environment
Even before the analytics system finishes everything,jobs that need a finished result carry on.
Live contextNow
09:06Monday morning
A
Analytics team09:06
Per-customer analytics results came out first.
Data readiness5 / 5
TransactionsDone
Customer dataDone
Service opsDone
External dataDone
Processed resultsDone
Downstream jobs now
Customer serviceCompleted
Operations statusCompleted
Integrated analyticsGenerating results
MONDAY, 10:15
This morning’s work seemed to be done.
Customer dataCompleted
Service operationsCompleted
Integrated analyticsCompleted
Analytics resultsCompleted
T
Transaction system team10:15
Some of the transaction records we sent this morning need corrections.We’re sending the corrected transaction data. Please apply it to the morning analytics results too.transaction_correction_1015.csv286 MB, newly created file
Does finished work have to start over from scratch?
INNORIX in Practice
Monday morning07:00
Monday 10:15
Unscheduled data arrives.
Some transaction records have been corrected.
The company already has analytics results built from the morning data. Resending all the source data and rerunning the entire analysis is not what it needs.
The existing analytics system already has a separate job for processing correction data.
ADDITIONAL DATA
New data needs its own processing conditions.
Scheduled morning work
594 GB
All source data
TransactionsCompleted
Customer dataCompleted
Service ops dataCompleted
External dataCompleted
Processed resultsCompleted
Earlier deliveries stay as they are
Extra correction job
286 MB
Newly created correction data
transaction_correction_1015.csvCreated 10:15
Correction-only Flow runs
Delivered to the correction system
What follows
10:15Correction data created
10:16Delivered to the correction environment
10:24Existing system processes the correction
10:28Corrected analytics results generated
10:29Results delivered to related services
Recurring work and exceptional workcan be set up in different ways.
Live contextNow
10:15Monday
T
Transaction system team10:15
Some of today’s transaction records need correcting. We’ve sent a correction file.
Data readiness5 / 5
TransactionsDone
Customer dataDone
Service opsDone
External dataDone
Processed resultsDone
Downstream jobs now
Customer serviceCompleted
Operations statusCompleted
Integrated analyticsApplying correction
Additional data
Correction data286 MB
Monday 11:40
How did today’s work actually go?
The operator reviews the results of the morning’s work.
Each job started and finished at a different time than planned, and an unscheduled extra job came up too.
MONDAY, 11:40
Scheduled work vs. what actually ran
07:0008:0009:0010:0011:0012:00
Customer service delivery
Operations data delivery
Integrated analytics delivery
Existing analytics system
Results follow-up delivery
Correction processing
Corrected results delivery
Scheduled start
Data delivery
Existing analytics processing
Extra correction job
INNORIX STUDIO
Today
Runs
Flow
Started
Completed
Status
Customer data
07:38
07:42
Completed
Service operations
07:51
08:13
Completed
Integrated analytics
08:47
08:49
Completed
Analytics results
09:06
09:50
Completed
Correction data
10:15
10:16
Completed
Corrected results
10:29
10:31
Completed
D
Data operations team
This morning’s data deliveries and the extra correction are both complete.
Today’s work didn’t follow the schedule,but each job ran once the data it needed was ready.
Live contextNow
11:40Monday
D
Data operations team11:40
We’ve confirmed the morning data delivery and the applied corrections.
Data readiness5 / 5
TransactionsDone
Customer dataDone
Service opsDone
External dataDone
Processed resultsDone
Downstream jobs now
Customer serviceCompleted
Operations statusCompleted
Integrated analyticsCompleted
THE NEXT DAY, Tuesday 07:32
Today was a little different from yesterday.
It’s the next morning.
There was less data to process than usual, and external data arrived early. All the required data from the five systems was ready at 7:32 a.m.
Jobs with no fixed run time don’t need to wait until the scheduled 8 a.m.
MONDAY & TUESDAY
Same work, different start times
07:0007:3008:0008:3009:00
Monday
Data prep
08:47
Tuesday
Data prepNext jobs
07:32
Original scheduled start
Monday
08:47
All required data ready
Ready 47 minutes later than scheduled
Tuesday
07:32
All required data ready
Ready 28 minutes earlier than scheduled
The run conditions are the same as yesterday.Only the time they were met has changed.
Live contextNow
07:32Tuesday morning
A
Analytics operations team07:32
All data was ready early today. Starting the analytics work.
Data readiness5 / 5
TransactionsDone
Customer dataDone
Service opsDone
External dataDone
Processed resultsDone
Downstream jobs now
Customer serviceCompleted
Operations statusCompleted
Integrated analyticsTransferring
Mon–Fri
The week that followed
The company’s work repeats every day. But the time its data was ready differed every day.
The dashed line marks 8 a.m., the original scheduled start time.
Day
Data ready
What was different
Monday
08:47
External and processed data delayed
Tuesday
07:32
All data ready early
Wednesday
08:13
More service usage logs
Thursday
09:17
External data transfer briefly paused
Friday
08:01
A fairly typical day
Even when the same work repeats,no two days are quite the same.
Live contextNow
Mon–FriThe week that followed
Data readiness5 / 5
TransactionsDone
Customer dataDone
Service opsDone
External dataDone
Processed resultsDone
Downstream jobs now
Customer serviceCompleted
Operations statusCompleted
Integrated analyticsCompleted
EVERY BUSINESS DAY
Tomorrow, too, the data won’t allbe ready at the same time.
Some work can start early,some has to wait for data that isn’t ready yet,and sometimes new, unscheduled work comes up.
Scheduled
Monday
08:00
Tuesday
08:00
Wednesday
08:00
Thursday
08:00
Friday
08:00
Actual data ready
Monday
08:47
Tuesday
07:32
Wednesday
08:13
Thursday
09:17
Friday
08:01
Work has a schedule.And it has conditions for when it can start.
Companies design their data movement with both in mind.
INNORIX
THE TECHNOLOGY BEHIND THIS STORY
Multi-Stage Flow
Set up and run downstream file transfers based on real business conditions, not just fixed schedules: data readiness, file creation and the completion of upstream transfers.