i'm using the latest version of dbt-duckdb (1.3.4) and still hitting this storage version error (i hoped torching the db, upgrading, and rebuilding would take care of it) -- is this a known issue? is there a workaround?

IO Error: Trying to read a database file with version number 38, but we can only read version 39. The database file was created with an older version of DuckDB. The storage of DuckDB is not yet stable; newer versions of DuckDB cannot read old database files and vice versa. The storage will be stabilized when version 1.0 releases. For now, we recommend that you load the database file in a supported version of DuckDB, and use the EXPORT DATABASE command followed by IMPORT DATABASE on the current version of DuckDB.

Duckdb’s file format is currently very fast moving and so even quite recent file types cannot be read by the most recently released software. See this issue on the dbt-duckdb repo: https://github.com/jwills/dbt-duckdb/issues/66 You have a couple of different options: • You could output to another data format, and then query from that format. DuckDB supports SQL queries to both read and write CSV (easy) or .parquet (more powerful) files, for example.

select * from 'mydata.csv'

• You could ask Josh Wills about it - it’s possible dbt-duckdb needs an update. I know duckdb have recently released 0.7.0 and though I don’t know exactly how python packages choose versions, it seems dbt-duckdb may still be on 0.5.0 https://github.com/jwills/dbt-duckdb/blob/master/setup.py

if i output to parquet can evidence read that in though?

Yes it can! We use the same syntax to do so that you can find on the duckdb docs: https://duckdb.org/docs/guides/import/parquet_import

was juuust about to ask re the Settings and all that, i don't see a CSV option?

What version of Evidence are you on?

great question 😆 -- i followed the Getting Started guide and npx'd the template project --- looks like that's 0.0.1 though -- i don't need the boilerplate/examples though if that's an old version i can trash this and pull a new one

Ahh I am so sorry. The csv connector has in fact not reached prime time yet!

evidence is v11

(We haven’t released it!) should come out some time this week at a guess!

sweeeet looking forward to it -- between that and josh's updates about to launch for duckdb 0.7.0 i'm sure i'll have a path forward soon -- and that makes sense as a work around, i'll see if i can get something of the right version in there as a placeholder to make it go

For example you can hook it up to the one shipped with the template repo: https://github.com/evidence-dev/template

yep i'm with you!

for what its worth, i'm using

external tables

with dbt-duckdb so it just drops all of tables into parquet.

sweet that's very good to know @lemon-cricket-39335 -- i'm building what is more or less an MDS in a box, so your learnings here are deeply appreciated!

fwiw i haven't figured out a good way to handle persistence yet