Uploading a csv via API

Question

Hello csv import is not working for me so i had to use the API for importing Here s a part of my code process env NODE TLS REJECT UNAUTHORIZED = 0 const supabase = createClient SUPABASE URL SUPABASE ANON KEY fs createReadStream transformed results test csv pipe csv on data async row => const newCompany = first name row first name || N A last name row last name || N A job title row job title || N A reworked job title row reworked job title || N A skills row skills || N A past company row past company || N A company city row company city || N A company linkedin url row company linkedin url || N A revenues row revenues parseInt row revenues null funding row funding parseInt row funding null company description row company description || N A company secondary industry row company secondary industry || N A tags row tags || N A const data error = await supabase from prospects insert newCompany select if error console log `Error inserting data $ error message ` else if data data length > 0 data forEach record => console log `Inserted company data $ JSON stringify record ` else console log No data inserted on end => console log CSV file successfully processed Terminal response in the first comment

Hugo Peran · Answer

Here s what my terminal is writing me back CSV file successfully processed Inserted company data id 58242d98 8107 4e01 b993 182d4dfad2eb created at 2023 05 25T18 11 59 618868+00 00 updated at 2023 05 25T18 11 59 618868+00 00 deleted at null first name christoph last name N A job title N A reworked job title N A business email N A personal email N A phone N A social url N A description N A keywords N A connections count null country N A city N A linkedin id N A skills N A past company N A rewards N A company name N A company domain N A company industry N A company address N A company country N A company city N A company founded null company size N A company linkedin url N A company phone N A company type N A company id N A email format N A revenues null funding null company description N A company secondary industry N A tags N A as you can see the first name is included and creates the row but the object send is not taking other values of the same rows How could I make it sure that the object being sent is full remember it s from a csv

garyaustin · Answer

Have you looked at using PSQL or pgAdmin thru the database port Assuming this is a one time setup and not an ongoing process you need automated thru the API

Hugo Peran · Answer

pgAdmin to mass import data This is sort of of one time setup for now but this code will be reused as mass update and row insertion through automation So pgAdmin option is not the best

Hugo Peran · Answer

Hello < garyaustin>

silentworks · Answer

Please don t mods If you were having a conversation with them before they will reply when they get the chance if they know the answer

Hugo Peran · Answer

Ok sorry I solved my problem it was a csv formating problem but weird thing happened My script imported 56k rows in 3 min but then it stopped at 200 people before the end Is the API rate limited

silentworks · Answer

It likely is maybe chunk them in intervals and see if that work