Sebastian Cantergiani
03/06/2024, 4:22 PMTechScholar Hub
03/06/2024, 6:59 PMUsageError: An error happened while creating the connection: type object 'ClickHouseDialect_http' has no attribute 'dbapi'.
Perhaps you meant to use the 'clickhouse' db
To find more information regarding connection: <https://jupysql.ploomber.io/en/latest/integrations/clickhouse.html>
To fix it:
Pass a valid connection string:
Example: %sql <postgresql://username:password@hostname/dbname>
For more details, see: <https://jupysql.ploomber.io/en/latest/connecting.html>
Sharon Machlis
03/10/2024, 3:48 AMSDOCA
03/13/2024, 4:21 PMMichel Bui
03/15/2024, 3:51 AMAlbert
03/27/2024, 9:11 PMNadira Shoketayeva
03/28/2024, 8:32 AM--with=transform_types,recrafts,image_types
--with transform_types --with recrafts --with image_types
--with transform_types,recrafts,image_types
Maciej Sawicki
03/28/2024, 2:05 PMtasks:
- source: tasks.slack.get_public_channels
product: products/raw/channels.json
- source: tasks.slack.tmp_fetch_channel
name: fetch-[[channel]]
product: products/raw/channels/[[channel]].txt
grid:
channel:
dotted_path: tasks.slack::get_channels_to_fetch
public_channels: products/raw/channels.json
relevant python code:
#pipelines/slack/tasks/slack.py
def get_public_channels(product):
#some code producing json file with channals list
def get_channels_to_fetch(public_channels):
with open(public_channels, "r") as f:
public_channels = json.load(f)
channels_to_ignore = ['some_channel_id1', 'some_channel_id2', 'some_channel_id_n']
return list(set(public_channels) - set(channels_to_ignore))
def tmp_fetch_channel(product, channel):
dir, file = os.path.split(os.path.abspath(product))
os.makedirs(dir, exist_ok=True)
with open(product, "w") as f:
f.write("test")
result:
$ls -lah products/raw/channels
total 480
drwxr-xr-x 62 msawicki staff 1.9K Mar 28 14:23 .
drwxr-xr-x 5 msawicki staff 160B Mar 28 14:23 ..
-rw-r--r-- 1 msawicki staff 296B Mar 28 14:23 .someid_1-29.txt.metadata
-rw-r--r-- 1 msawicki staff 296B Mar 28 14:23 .someid_2-23.txt.metadata
-rw-r--r-- 1 msawicki staff 296B Mar 28 14:23 .someid_3-7.txt.metadata
(..)
-rw-r--r-- 1 msawicki staff 4B Mar 28 14:23 someid_1-29.txt
-rw-r--r-- 1 msawicki staff 4B Mar 28 14:23 someid_2-23.txt
-rw-r--r-- 1 msawicki staff 4B Mar 28 14:23 someid_3-7.txt
expected output:
$ls -lah products/raw/channels
total 480
drwxr-xr-x 62 msawicki staff 1.9K Mar 28 14:23 .
drwxr-xr-x 5 msawicki staff 160B Mar 28 14:23 ..
-rw-r--r-- 1 msawicki staff 296B Mar 28 14:23 .someid_1.txt.metadata
-rw-r--r-- 1 msawicki staff 296B Mar 28 14:23 .someid_2.txt.metadata
-rw-r--r-- 1 msawicki staff 296B Mar 28 14:23 .someid_3.txt.metadata
(..)
-rw-r--r-- 1 msawicki staff 4B Mar 28 14:23 someid_1.txt
-rw-r--r-- 1 msawicki staff 4B Mar 28 14:23 someid_2.txt
-rw-r--r-- 1 msawicki staff 4B Mar 28 14:23 someid_3.txt
My question are:
1. why the integer suffix are added to the file names?
2. how to get rid of them?Maciej Sawicki
03/28/2024, 2:09 PMpublic_channels: products/raw/channels.json)
, so it refers to tasks.slack.get_public_channels
product instead of hardcoding products/raw/channels.json
?ray
03/28/2024, 3:10 PMMatthew Madahar
03/30/2024, 6:00 PMMatan (Mathiverse)
04/01/2024, 4:52 PMChan Jong Na
04/03/2024, 4:41 AMJan-Hendrik MĂĽller
04/04/2024, 10:25 AMDavid Tang
04/04/2024, 10:50 AMploomber-cloud
CLI.
Is there any settings that I can set to change the resources provisioning? currently it defaults to 0.5CPU, and the --help flag does not show any options to change this. (ploomber-cloud 0.1.20)Dubba Srikanth Reddy
04/04/2024, 5:33 PMDobromir Chankov
04/07/2024, 8:08 AM# pipeline_1.yaml
tasks:
- source:
- aop_xml_schema_data_classes.py
- aop-wiki-xml-2024-01-01.xml
- product: parsing_helpers.py
- source:
- parsing_helpers.py
- template_aop.txt
- template_ke.txt
- template_ker.txt
- product: tests_with_paperqa_3.ipynb
- source: tests_with_paperqa_3.ipynb
- product: tests_with_paperqa_4.ipynb
What I'm doing wrong? :)
Many thanks for your comments!!!Dobromir Chankov
04/07/2024, 8:20 AM2024-04-07T07:44:52.078000 - INFO Retrieving image manifest <http://XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm|XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm>
2024-04-07T07:44:52.079000 - INFO Retrieving image <http://XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm|XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm> from registry <http://XYZ.dkr.ecr.REGION.amazonaws.com|XYZ.dkr.ecr.REGION.amazonaws.com>
2024-04-07T07:44:52.889000 - INFO Retrieving image manifest <http://XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm|XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm>
2024-04-07T07:44:52.889000 - INFO Returning cached image manifest
2024-04-07T07:44:52.977000 - INFO Built cross stage deps: map[]
2024-04-07T07:44:52.978000 - INFO Retrieving image manifest <http://XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm|XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm>
2024-04-07T07:44:52.978000 - INFO Returning cached image manifest
2024-04-07T07:44:52.978000 - INFO Retrieving image manifest <http://XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm|XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm>
2024-04-07T07:44:52.978000 - INFO Returning cached image manifest
2024-04-07T07:44:52.979000 - INFO Executing 0 build triggers
2024-04-07T07:44:52.979000 - INFO Building stage '<http://XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm|XYZ.dkr.ecr.REGION.amazonaws.com/docker-hub/library/python:3.11-slim-bookworm>' [idx: '0', base-idx: '-1']
2024-04-07T07:44:52.980000 - INFO Checking for cached layer <http://XYZ.dkr.ecr.REGION.amazonaws.com/prod-REGION-build-image-cache:714c120d8af42d6c6dbeafbafe20a6cf82992d8fbea053d0c4a12d1c7a8e700f|XYZ.dkr.ecr.REGION.amazonaws.com/prod-REGION-build-image-cache:714c120d8af42d6c6dbeafbafe20a6cf82992d8fbea053d0c4a12d1c7a8e700f>...
2024-04-07T07:44:53.182000 - INFO Using caching version of cmd: RUN pip install voila ipywidgets ipykernel 'nbformat<=5.9.2' --no-cache-dir
2024-04-07T07:44:53.182000 - INFO Checking for cached layer <http://XYZ.dkr.ecr.REGION.amazonaws.com/prod-REGION-build-image-cache:ba44c23ee46780fdd89c84f0fbbf82965f3f84f822d8fca7353f71256eeedd2b|XYZ.dkr.ecr.REGION.amazonaws.com/prod-REGION-build-image-cache:ba44c23ee46780fdd89c84f0fbbf82965f3f84f822d8fca7353f71256eeedd2b>...
2024-04-07T07:44:53.285000 - INFO No cached layer found for cmd RUN pip install -r requirements.txt --no-cache-dir
2024-04-07T07:44:53.287000 - INFO Unpacking rootfs as cmd COPY requirements.txt /srv/ requires it.
2024-04-07T07:44:56.680000 - INFO RUN pip install voila ipywidgets ipykernel 'nbformat<=5.9.2' --no-cache-dir
2024-04-07T07:44:56.680000 - INFO Found cached layer, extracting to filesystem
2024-04-07T07:45:00.333000 - INFO WORKDIR /srv
2024-04-07T07:45:00.333000 - INFO Cmd: workdir
2024-04-07T07:45:00.333000 - INFO Changed working directory to /srv
2024-04-07T07:45:00.333000 - INFO No files changed in this command, skipping snapshotting.
2024-04-07T07:45:00.334000 - INFO COPY requirements.txt /srv/
2024-04-07T07:45:00.334000 - INFO Taking snapshot of files...
2024-04-07T07:45:00.340000 - INFO RUN pip install -r requirements.txt --no-cache-dir
2024-04-07T07:45:00.340000 - INFO Initializing snapshotter ...
2024-04-07T07:45:00.340000 - INFO Taking snapshot of full filesystem...
2024-04-07T07:45:08.308000 - INFO Cmd: /bin/sh
2024-04-07T07:45:08.308000 - INFO Args: [-c pip install -r requirements.txt --no-cache-dir]
2024-04-07T07:45:08.308000 - INFO Running: [/bin/sh -c pip install -r requirements.txt --no-cache-dir]
2024-04-07T07:45:08.756000 - ERROR: Invalid requirement: 'tasks:' (from line 3 of requirements.txt)
2024-04-07T07:45:09.014000 - error building image: error building stage: failed to execute command: waiting for process to exit: exit status 1
2024-04-07T07:45:32.198000 - The container failed with status 1
Sharon Machlis
04/07/2024, 4:05 PMIsabel Pietri
04/07/2024, 6:24 PMSharon Machlis
04/07/2024, 9:20 PMSharon Machlis
04/07/2024, 9:21 PMChan Jong Na
04/09/2024, 3:09 PM2024-04-09T15:07:37.777000 - ERROR: packages/rag-conversation is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with bzr+http, bzr+https, bzr+ssh, bzr+sftp, bzr+ftp, bzr+lp, bzr+file, git+http, git+https, git+ssh, git+git, git+file, hg+file, hg+http, hg+https, hg+ssh, hg+static-http, svn+ssh, svn+http, svn+https, svn+svn, svn+file).
I have been installing them by writing
-e packages/rag-conversation
in requirements.txt
Treya
04/10/2024, 3:28 AMCREATE DATABASE myCompanyDB;
error:
RuntimeError: (singlestoredb.exceptions.OperationalError) 1044: Forwarding Error (node-3bf5efad-649f-4367-9878-91bb7de81630-master-0.svc-3bf5efad-649f-4367-9878-91bb7de81630:3306): Access denied for user '6f16db91-d948-426a-a3d3-c9beeeb3bf08'@'%' to database 'myCompanyDB'
[SQL: CREATE DATABASE myCompanyDB;]
(Background on this error at: <https://sqlalche.me/e/20/e3q8>)
If you need help solving this issue, send us a message: <https://ploomber.io/community>
Yogeshkumar R
04/10/2024, 5:52 AMMateusz Waćkowski
04/11/2024, 5:57 PMNathan Seltzer
04/15/2024, 3:14 PMMatteo Ferrini
04/18/2024, 9:05 AMFlavio Martins
04/19/2024, 1:35 PMChan Jong Na
04/20/2024, 4:19 PM