This message was deleted.
# ask-anything
s
This message was deleted.
e
is "CAPSL-NASUNI" a product of the data_pull task? based on your pipeline.yaml it looks like it. but based on the error message, it looks like you're trying to read it (so it might be an input?)
l
it is not
Many of our servers begin with the name CAPSL but I am not specifying this in my data_pull task so I am not sure how Ploomber was able to detect it
e
so it sounds like this is an input data file right? in such case, it should not be part of the
product
section
l
it's an output data file
data_pull's job is to write a csv file to the I drive
e
can you share the full traceback? I'm a bit confused about the error
l
sure, need a few minutes
Loading pipeline... Executing: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [0049<0000, 6.14s/cell] Executing: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [0009<0000, 1.23cell/s] Executing: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [0012<0000, 1.26cell/s] Executing: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11/11 [0009<0000, 1.20cell/s] Executing: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [0151<0000, 14.00s/cell] Building task 'data_table_construction': 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [0313<0000, 38.64s/it]
e
looks like the pipeline is running without errors? I don't see any traceback
or error messages
l
sorry, got caught up in stuff, one sec
that is the weird part, the pipeline finishes, but then I get
Copy code
============================================================================================== DAG build failed ===============================================================================================
----------------------------------- NotebookRunner: pull_weekly_tcd_raw_data -> MetaProduct({'data': File('\\\\CAP...\\vessel.csv'), 'nb': File('product...w_data.ipynb')}) ------------------------------------
-------------------------- C:\Users\lfunderburk\OneDrive - Seaspan Corporation\Documents\GitHub\maritime_technology_weekly_tcd\weeklytcd\pipeline\scripts\pull_weekly_tcd_raw_data.py --------------------------
Traceback (most recent call last):
  File "C:\Users\lfunderburk\Anaconda3\envs\venv\lib\site-packages\ploomber\tasks\abc.py", line 583, in _build
    self._post_run_actions()
  File "C:\Users\lfunderburk\Anaconda3\envs\venv\lib\site-packages\ploomber\tasks\abc.py", line 357, in _post_run_actions
    self.product.metadata.update(
  File "C:\Users\lfunderburk\Anaconda3\envs\venv\lib\site-packages\ploomber\products\metadata.py", line 359, in update
    p.metadata.update(source_code, params)
  File "C:\Users\lfunderburk\Anaconda3\envs\venv\lib\site-packages\ploomber\products\metadata.py", line 263, in update
    self._product.save_metadata(data)
  File "C:\Users\lfunderburk\Anaconda3\envs\venv\lib\site-packages\ploomber\products\file.py", line 74, in save_metadata
    self._path_to_metadata.write_text(json.dumps(metadata))
  File "C:\Users\lfunderburk\Anaconda3\envs\venv\lib\pathlib.py", line 1255, in write_text
    with self.open(mode='w', encoding=encoding, errors=errors) as f:
  File "C:\Users\lfunderburk\Anaconda3\envs\venv\lib\pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "C:\Users\lfunderburk\Anaconda3\envs\venv\lib\pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '\\\\CAPSL-NASUNI\\path-to-files\.vessel.csv.metadata'
this happens for every step in the pipeline
e
ok, this looks like a bug. is this the actual value that you have in your pipeline.yaml?
data: I:/path-to-files/file_name.csv
? I can dig into it, but as a workaround, you can change the products to have relative paths and move them with another script to the
I:
drive
l
Thank you Eduardo. Not posting the full path for security reasons
Ok, can use that as a workaround
Would you need the full path?
e
not necessary. I'm guessing it has the
I:/something/another.extension
format, right?
l
yes
e
did the workaround fix the issue?
l
I ran into a permissions issue when I tried it.
Likely due to security configuration of my company. Will try a couple more things and inquire internally.
e
ok