Hi, I'm running into the following issue

[Errno 7] Argument list too long: 'dbt'

when running the

upload-source-freshness

operation. From what I've seen from previous tickets it was supposed to be solved, is there anything I can do regarding this? https://github.com/elementary-data/elementary/pull/850 Versions I'm using: python elementary 0.13.2 dbt elementary 0.13.2 edr version 0.13.2 The full log:

edr run-operation upload-source-freshness --project-dir .
    ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - <https://bit.ly/slack-elementary>

2024-01-17 12:38:57 — INFO — Running with edr=0.13.2
Uploading source freshness results |⚠︎                                       | (!) 0/3 [0%] in 0.1s (0.00/s)
Traceback (most recent call last):
  File "/usr/local/bin/edr", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elementary/cli/cli.py", line 67, in invoke
    return super().invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elementary/operations/cli.py", line 57, in upload_source_freshness
    UploadSourceFreshnessOperation(config).run()
  File "/usr/local/lib/python3.11/site-packages/elementary/operations/upload_source_freshness.py", line 26, in run
    self.upload_results(results, metadata)
  File "/usr/local/lib/python3.11/site-packages/elementary/operations/upload_source_freshness.py", line 72, in upload_results
    dbt_runner.run_operation(
  File "/usr/local/lib/python3.11/site-packages/elementary/clients/dbt/dbt_runner.py", line 154, in run_operation
    success, command_output = self._run_command(
                              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/elementary/clients/dbt/dbt_runner.py", line 82, in _run_command
    result = subprocess.run(
             ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.11/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 7] Argument list too long: 'dbt'

@Leonardo Melo, unfortunately I don't have a solution out-of-the-box. I'm working on a fix right now and will update shortly. Please note that it'll only be available in the upcoming version, most likely next week.

No problem, thats alright and thanks for letting me know!

Here it is. It'll allow you to use

--rows-per-insert

and re-configure the amount of rows that are uploaded per insert. I've also lowered the default from 100 to 25. It'll be reviewed by the team soon.

Perfect, looks good. Thanks!