Hi, about on failure . i did some tests and i don't exactly understand the behavior .
_case 1: _
replication with a table source that doesn't exists. - result
| Error 1146 (42S02): Table 'bi_Assets_1' doesn't exist
bi_Assets_1 |
bi_Assets_1 | fatal:
12:05PM INF Sling Replication Completed in 16s |DASH -> target_stg | 28 Successes | 1 Failures
12:05PM WRN failed streams: ["bi_Assets_1"]
fatal:
~ failure running replication (see docs @
https://docs.slingdata.io/sling-cli)
Replication failure (1 streams failed).
it didn't fail in the middle or stopped the replication.
also i write the log to a table with '{run.status}', and {run.stream.table}
and this stream doesn't exist in the log, I expected to have the source stream that doesn't exist. with an error.
_case 2: _
I went on and created a pipeline that runs this replication, with on_failure: abort
it did the same thing. didn't abort . didn't fail the run
bi_Assets_1 | fatal:
bi_Assets_1 | ~ SQL Error
bi_Assets_1 | Error 1146 (42S02): Table 'Assets_1' doesn't exist
12:15PM INF Sling Replication Completed in 16s | DASH -> target_stg | 28 Successes | 1 Failures
12:15PM WRN failed streams: ["bi_Assets_1"]
fatal:
~ error running replication
Replication failure (1 streams failed)
no log, no abort.
_case 3: _
I created a loop over 4 tables in the target DB and send the names as parameters to the replication
streams:
- "{loop.value.table_name}"
i put there a table that doesn't exist as stream in the replication .
I didn't get any error. the process ignored it completely.
I would expect at least a warning about the stream I try to run that doesn't exist.
How exactly should the abort work?
I thought it will stop the process all together. Especially if it is a pipeline with abort.