https://slingdata.io logo
Hi, about on failure . i did some tests
# sling-help
i
Hi, about on failure . i did some tests and i don't exactly understand the behavior . _case 1: _ replication with a table source that doesn't exists. - result | Error 1146 (42S02): Table 'bi_Assets_1' doesn't exist bi_Assets_1 | bi_Assets_1 | fatal: 12:05PM INF Sling Replication Completed in 16s |DASH -> target_stg | 28 Successes | 1 Failures 12:05PM WRN failed streams: ["bi_Assets_1"] fatal: ~ failure running replication (see docs @ https://docs.slingdata.io/sling-cli) Replication failure (1 streams failed). it didn't fail in the middle or stopped the replication. also i write the log to a table with '{run.status}', and {run.stream.table} and this stream doesn't exist in the log, I expected to have the source stream that doesn't exist. with an error. _case 2: _ I went on and created a pipeline that runs this replication, with on_failure: abort it did the same thing. didn't abort . didn't fail the run bi_Assets_1 | fatal: bi_Assets_1 | ~ SQL Error bi_Assets_1 | Error 1146 (42S02): Table 'Assets_1' doesn't exist 12:15PM INF Sling Replication Completed in 16s | DASH -> target_stg | 28 Successes | 1 Failures 12:15PM WRN failed streams: ["bi_Assets_1"] fatal: ~ error running replication Replication failure (1 streams failed) no log, no abort. _case 3: _ I created a loop over 4 tables in the target DB and send the names as parameters to the replication streams: - "{loop.value.table_name}" i put there a table that doesn't exist as stream in the replication . I didn't get any error. the process ignored it completely. I would expect at least a warning about the stream I try to run that doesn't exist. How exactly should the abort work? I thought it will stop the process all together. Especially if it is a pipeline with abort.
f
hi @itamar s what version are you using? can you try with lates 1.5.20?
A replication is not fail-fast by design. So it will run all streams and output the failed stream at the end.
for case 3, yes if it didn't match any stream, it emits a trace debug
skipping stream %s since it is not selected
, but yes agreed a warning that no stream matched would be helpful. Adding that