https://www.boundaryml.com logo
Powered by
🟡 Issues with BAML Playground and Huggingface Inf...
# questions
a
Hi Everyone! I am trying out the huggingface inference API, and run into issues with BAML playground. This is the client definition (btw the BAML doc is out of date - containing the obselete endpoint and api_key name)
Copy code
client<llm> HFModel {
  provider openai-generic
  options {
    model "moonshotai/Kimi-K2-Thinking:novita"
    base_url "https://router.huggingface.co/v1/"
    api_key env.HF_TOKEN
  }
}
Running it will result in an error with unclear error message (see pic2) But I copied the cURL (shown below) provied by BAML, paste it into terminal after exporting my HF_TOKEN, then it works... (see pic 2)
Copy code
curl -X POST 'https://router.huggingface.co/v1/chat/completions' -H "authorization: Bearer $HF_TOKEN" -H "content-type: application/json" -d "{
  \"model\": \"moonshotai/Kimi-K2-Thinking:novita\",
  \"messages\": [
    {
      \"role\": \"system\",
      \"content\": \"Answer the following question in less than 10 words:\nWhat is the capital of France?\n\n\"
    }
  ]
}"
I have provided the HF_TOKEN via the 
Manages API KEYs
in BAML (see pic 3) Got 2 questions hope you could kindly help out 1. how do I get more meaningful error message from BAML playground? The error message has always been vague, but I used to be able to figure the problem out by running the cURL instead. But now the curl is working, so I run out of way to debug 😢 2. What should I try to do to get it work in the playground now? https://cdn.discordapp.com/attachments/1253172394345107466/1461930235951059161/image.png?ex=697ad8a1&is=69798721&hm=a75105a40f71fd9537ee7f0be549bced7c908ae1c99a66d447a90cbca9b2812f& https://cdn.discordapp.com/attachments/1253172394345107466/1461930236244529405/image.png?ex=697ad8a1&is=69798721&hm=73fe9a0183ebd62c26ed1f4097b09aba2d1576b4b6461ef9423eee7ae689abd7& https://cdn.discordapp.com/attachments/1253172394345107466/1461930236559098039/image.png?ex=697ad8a1&is=69798721&hm=25656f881f9f95f2d141d5ce59e556a27b7fa32e133ed9bae27e8ac96bdafab5&
Not sure if this helps investigation, calling baml via python also works.
v
that acutally helps a lot!
so whats happening here is almost def a CORS error (Which sadly the browser prevents from being any better 🙁 )
but you can confirm this, by opening developer tools in vscode
and looking at the network tab
if thats the case instead you do:
(the fetch API gives us a standard fetch error - similar to the one you get when network is down or when CORS errors since the browser tries to be safe and prevent website code from knowing its a CORS problem)
a
@Vaibhav (BAML 🐑) Thanks for the prompt reply! Unfortunately I don't have enough knowledge on network to figure out if it's s CORS error ... I just checked the setting, and the bypass CORS is already ticked by default. I guess it means it's not a CORS issue? Also the first screenshot has a unauthorised 401 error, does it suggest there is authentication issue with the API key? https://cdn.discordapp.com/attachments/1461930236873801860/1461939336223260766/image.png?ex=696c609b&is=696b0f1b&hm=a2deb99bed33478ac29bd903dbd7ff8005c72679d334241b0f9530df95380f17& https://cdn.discordapp.com/attachments/1461930236873801860/1461939336537837801/image.png?ex=696c609b&is=696b0f1b&hm=b7c8a99cefee39c4101df34cc0d3478b924d38bbd297fea93cb5955ec9b543ad&
v
a
v
want to hop on Office hours for a second and screenshare?
may be faster
a
yeh that'd be great, thanks man, let me register a office hours account first
It doens't seem to allow personal account? Do I even need to register or I can hop on a link you share?
v
nope no account needed!
a
great, let me know how I can join then
v
a
Thank you so much for fixing it for me in such short period of time@Vaibhav (BAML 🐑) For reference if anyone has the same issue, the solution for my case was turning off the VSCode Proxy and restart the playground. https://cdn.discordapp.com/attachments/1461930236873801860/1461944466016309384/image.png?ex=696c6562&is=696b13e2&hm=27ae5b9914014aa2eef30dce8a094738fbe66da29661c39c1387c6891ad7f3d9&
j
I have the same problem, tried both VSCode proxy on and off. Cursor devtool show CORS error.
v
Hmm. Can you try making it on, then closing the playground and opening it again?
PreviousNext
Powered by