https://linen.dev logo
Join Slack
Powered by
[2025-12-05 11:15:19] `INFO` - `Error inserting fi...
# ask-ai
a
[2025-12-05 111519] 
INFO
- 
Error inserting file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Error updating file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Converted and uploaded sheet 'EquityChanges' from /home/dtmsbi_test/BK_techouse/source/Expanded_Financial_Reporting_Dataset_Client5.7.xlsx to /home/dtmsbi_test/BK_techouse/source_csv/Expanded_Financial_Reporting_Dataset_Client5.7_EquityChanges.csv
source=task.stdout [2025-12-05 111519] 
INFO
- 
Error inserting file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Error updating file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Converted and uploaded sheet 'ForexTransactions' from /home/dtmsbi_test/BK_techouse/source/Expanded_Financial_Reporting_Dataset_Client5.7.xlsx to /home/dtmsbi_test/BK_techouse/source_csv/Expanded_Financial_Reporting_Dataset_Client5.7_ForexTransactions.csv
source=task.stdout [2025-12-05 111519] 
INFO
- 
Error inserting file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Error updating file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Converted and uploaded sheet 'BalanceSheet' from /home/dtmsbi_test/BK_techouse/source/Expanded_Financial_Reporting_Dataset_Client5.7.xlsx to /home/dtmsbi_test/BK_techouse/source_csv/Expanded_Financial_Reporting_Dataset_Client5.7_BalanceSheet.csv
source=task.stdout [2025-12-05 111519] 
INFO
- 
Error inserting file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Error updating file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Converted and uploaded sheet 'ProfitAndLoss' from /home/dtmsbi_test/BK_techouse/source/Expanded_Financial_Reporting_Dataset_Client5.7.xlsx to /home/dtmsbi_test/BK_techouse/source_csv/Expanded_Financial_Reporting_Dataset_Client5.7_ProfitAndLoss.csv
source=task.stdout [2025-12-05 111519] 
INFO
- 
Error inserting file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Error updating file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Converted and uploaded sheet 'NewCashFlow' from /home/dtmsbi_test/BK_techouse/source/Expanded_Financial_Reporting_Dataset_Client5.7.xlsx to /home/dtmsbi_test/BK_techouse/source_csv/Expanded_Financial_Reporting_Dataset_Client5.7_NewCashFlow.csv
source=task.stdout [2025-12-05 111519] 
INFO
- 
Error inserting file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Error updating file processing record: current transaction is aborted, commands ignored until end of transaction block
source=task.stdout [2025-12-05 111519] 
INFO
- source=task.stdout [2025-12-05 111519] 
INFO
- 
Processed file details: [{'processedFileId': None, 'WorkbookID': 1, 'parentFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7.xlsx', 'processedCsvFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7_ChartOfAccounts.csv'}, {'processedFileId': None, 'WorkbookID': 1, 'parentFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7.xlsx', 'processedCsvFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7_GeneralLedger.csv'}, {'processedFileId': None, 'WorkbookID': 1, 'parentFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7.xlsx', 'processedCsvFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7_JournalEntries.csv'}, {'processedFileId': None, 'WorkbookID': 1, 'parentFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7.xlsx', 'processedCsvFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7_JournalEntryLines.csv'}, {'processedFileId': None, 'WorkbookID': 1, 'parentFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7.xlsx', 'processedCsvFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7_CashFlow.csv'}, {'processedFileId': None, 'WorkbookID': 1, 'parentFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7.xlsx', 'processedCsvFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7_EquityChanges.csv'}, {'processedFileId': None, 'WorkbookID': 1, 'parentFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7.xlsx', 'processedCsvFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7_ForexTransactions.csv'}, {'processedFileId': None, 'WorkbookID': 1, 'parentFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7.xlsx', 'processedCsvFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7_BalanceSheet.csv'}, {'processedFileId': None, 'WorkbookID': 1, 'parentFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7.xlsx', 'processedCsvFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7_ProfitAndLoss.csv'}, {'processedFileId': None, 'WorkbookID': 1, 'parentFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7.xlsx', 'processedCsvFileName': 'Expanded_Financial_Reporting_Dataset_Client5.7_NewCashFlow.csv'}]
source=task.stdout [2025-12-05 111519] 
INFO
- 
Error storing processed file details: [Errno 2] No such file or directory: '/home/test-machine/airflow/dags/processed_file_details.json'
source=task.stdout [2025-12-05 111519] 
INFO
- 
Error moving file /home/dtmsbi_test/BK_techouse/source/Expanded_Financial_Reporting_Dataset_Client5.7.xlsx to archive: Failure
source=task.stdout [2025-12-05 111519] 
INFO
- 
[chan 0] sftp session closed.
source=paramiko.transport.sftp loc=sftp.py:169 [2025-12-05 111519] 
INFO
- 
Done. Returned value was: None
source=airflow.task.operators.airflow.providers.standard.operators.python.PythonOperator loc=python.py:215 [2025-12-05 111519] 
INFO
- 
Task instance in success state
source=task.stdout [2025-12-05 111519] 
INFO
- 
Previous state of the Task instance: TaskInstanceState.RUNNING
source=task.stdout [2025-12-05 111519] 
INFO
- 
Task operator:<Task(PythonOperator): file_processing_workflow>
source=task.stdout
k
These logs look like they’re from an Airflow PythonOperator task that: - Reads an XLSX workbook - Generates multiple CSVs (one per sheet) - Tries to: - Insert/update “file processing” records in a database - Store a JSON summary at 
/home/test-machine/airflow/dags/processed_file_details.json
- Move the source XLSX to an archive directory Key points from the log snippet: 1. Database errors Messages like: > 
Error inserting file processing record: current transaction is aborted, commands ignored until end of transaction block
> 
Error updating file processing record: current transaction is aborted, commands ignored until end of transaction block
mean that an earlier SQL statement in the same transaction failed, and the connection stayed in an aborted state. Every subsequent statement on that connection is then refused until the transaction is rolled back. From the provided Airbyte docs, there’s no specific guidance on this application-level pattern; error-handling requirements only state that destinations must not have silent errors and should log them or fail the sync, but the exact DB transaction handling is up to your implementation.[[Error handling](https://docs.airbyte.com/platform/connector-development/partner-certified-destinations#error-handling)] So from the knowledge sources, all that can be said is: - Your code is correctly logging these errors (no “silent” failure), which aligns with Airbyte’s general error‑handling guidance. - The root cause of the DB failure that triggered the aborted transaction is not visible in this snippet, and the Airbyte knowledge base does not document this specific workflow or error. 2. File system error This line: > 
Error storing processed file details: [Errno 2] No such file or directory: '/home/test-machine/airflow/dags/processed_file_details.json'
indicates that the script is trying to write to a file path under 
/home/test-machine/airflow/dags/
that doesn’t exist (directory or file). The Airbyte File source connector documentation only describes how to use external files (S3/GCS/HTTPS/SFTP) as a source; it doesn’t cover this Airflow-local JSON logging pattern, so there’s no Airbyte-specific fix documented.[[File source](https://docs.airbyte.com/integrations/sources/file)] 3. Task marked successful despite errors Finally: > 
Done. Returned value was: None
> 
Task instance in success state
shows the Python callable finished without raising an exception, so Airflow marked the task as 
success
even though it logged errors. This is an application-level design choice; Airbyte’s guidance for destinations is that they should fail if they can’t reliably communicate or log errors, especially for publish destinations, but it doesn’t define behavior for your custom preprocessing job.[[Error handling](https://docs.airbyte.com/platform/connector-development/partner-certified-destinations#error-handling)] --- ### What the Airbyte docs can and can’t tell you Within the provided knowledge sources: - There is no documentation about this specific “file_processing_workflow” job, its database schema, or the exact cause of the initial DB error. - The docs do confirm: - Connectors and partner destinations must log errors rather than silently ignore them.[[Error handling](https://docs.airbyte.com/platform/connector-development/partner-certified-destinations#error-handling)] - The File source connector expects files in storage systems like S3/GCS/HTTPS/SFTP and does not mention local JSON “processed_file_details” tracking.[[File source](https://docs.airbyte.com/integrations/sources/file)] Given that, I cannot reliably say why your DB transaction initially failed or how your Airflow DAG is intended to behave beyond what’s visible in the logs. --- ### Practical next steps (based on logs only) These are generic, not Airbyte-documented steps: 1. Find the first DB error in the task logs (it will precede the “current transaction is aborted” messages) to see the real cause. 2. Adjust the DB code to: - Roll back on error, then either: - Retry the insert/update, or - Fail the task explicitly. 3. Ensure the output directory exists before writing 
processed_file_details.json
(e.g., create 
/home/test-machine/airflow/dags/
or change the path). 4. Decide whether the task should fail on these errors: - If tracking / archiving is critical, raise an exception so Airflow marks the task as failed. - If they’re non-critical, at minimum keep logging them (which you already do), consistent with the “no silent errors” principle.[[Error handling](https://docs.airbyte.com/platform/connector-development/partner-certified-destinations#error-handling)] If you can share the part of the log where the first database error appears (before the “current transaction is aborted” messages), I can help interpret that more precisely, but the Airbyte documentation you provided does not itself describe this particular error pattern.
10 Views
Open in Slack
PreviousNext
Powered by