https://toitlang.org/ logo
Join Discord
Powered by
How to serve HTML, CSS, Javascript files
# help
h
Hi, back in my Arduino days (two days ago) I had a webserver running on my ESP32-S2. It served one HTML file and a CSS file and Javascript files. There was also JSON going back and forth. The website was used for adjusting settings in my ESP software, for instance choose a colour for the attached LEDs. I am struggling to find documentation on how to achieve the same with Toit. Including the HTML as a string in my Toit program is not only uncool but even inconvenient. What's the trick here? Do you know? I read that people have SD-cards connected to their ESP32, but I do not have that and would like to avoid adding hardware to my PCB if possible. I could get the files from an external webserver but I would rather not depend on that, it opens up a can of worms (i.e. proxies, firewalls, etc.) and I would like the device to be mostly self-sufficient. Thank for any tips or thoughts on the matter.
k
Maybe something like 'assets' would work for you?
We'll probably need to write up how to do this easily through Jaguar.
h
Thanks Kasper, that will do. A bit more work than just "jag watch programname.toit", but by the looks of it, it can be easily automated. I'll give it a go.
k
@hhv0001 You're getting into the deep end. Love it 🤩
h
😀 I see there's an --assets argument to "jag run" and "jag watch"! That's looking great. These assets do not change often.
So far I have this:
Copy code
else if request.path == "/index.html":
      writer.headers.set "Content-Type" "text/html"
      writer.write assets.decode["index.html"]
and it works!
k
Awesome!
h
Btw I get this compile error when trying to close the ResponseWriter: Class 'ResponseWriter' does not have any method 'close' writer.close Shall I make a bug report for that or is it my bad? (I'm not a OO-guru but shouldn't the class ResponseWriter have the class Writer as a parent, and not Object?)
I tried writer.close-write and writer.close-writer too...
k
@hhv0001 That sounds a bit weird. The 
ReponseWriter
class does have a 
close
method.
Which version of 
pkg-http
are you using? You can probably tell by looking at your 
package.lock
file.
h
It says : packages: pkg-http: url: github.com/toitlang/pkg-http name: http version: 1.9.3 hash: 108c436cc990535f5d70c380ef68081c38840f4c
That server.toit is the example I copied as a starting point.
k
It looks like you're stuck on a slighty old version.
h
Ah.
k
Try 
jag pkg uninstall http
and then reinstall it using 
jag pkg install http
.
Not sure how you ended up with a 
1.x
version. When / how did you install this in the first place?
(maybe we have some outdated documentation somewhere)
h
I had to uninstall websocket as well and now I have version 2.3.4, no more errors. Thanks! I have probably installed that older version of http a few months ago, on my first Toit-attempt.
k
I think the websocket support is now included in the http package. Does that sound right @erikcorry?
e
Yes
Yes
k
Yes
h
Thanks guys.
k
Yes
😋
Thanks for giving this a spin!
h
It's not terribly important for me atm but I am running into something I can't fix. I have started off with the webserver example and added assets as I mentioned above. That works reasonably well, except when I load a page that has links to CSS and Javascript in it, then the requests for these CSS and Javascript files take additional 30 seconds to download. I am also seeing this: DEBUG: incoming request {peer: 10.1.0.166:56947, path: /index.html} Handling request for /index.html ****************************************************************************** Decoding by 
jag
, device has version ****************************************************************************** EXCEPTION error. Connection closed
0: tcp-write_ /net/modules/tcp.toit:199:3 1: TcpSocket.write /net/modules/tcp.toit:165:16 2: Writer.write /writer.toit:39:23 3: ChunkedWriter.write-header_ /chunked.toit💯13 4: ChunkedWriter.write /chunked.toit:77:5 5: ResponseWriter.write /server.toit:202:18 6: main. webserver.toit:122:14 7: Server.run-connection_. /server.toit:164:19 8: catch. /core/exceptions.toit:124:10 9: catch /core/exceptions.toit:122:1 10: catch /core/exceptions.toit:85:10 11: Server.run-connection_ /server.toit:163:9 12: Server.listen.... /server.toit:113:26 13: catch. /core/exceptions.toit:124:10 14: catch /core/exceptions.toit:122:1 15: catch /core/exceptions.toit:97:10 16: Server.listen... /server.toit:112:18 17: Server.listen.. /server.toit:109:38 18: Server.listen. /server.toit:129:37 19: Server.listen /server.toit:85:3 20: main webserver.toit:111:10 ******************************************************************************
INFO: Internal Server error - Connection closed {peer: 10.1.0.166:56947, path: /index.html} DEBUG: connection ended {peer: 10.1.0.166:56947, reason: Connection closed} and Handling request for /add-color-wheel.js DEBUG: connection ended {peer: 10.1.0.166:56939, reason: DEADLINE_EXCEEDED} The trouble is not with the assets I think. When I replace looking up an asset with HTML or CSS I see the same behaviour. Any pointers on how to fix this or examine this are welcome. I don't think I fully understand how server.listen in the main: function works f.i.
k
@hhv0001 You may benefit from allowing your http server to serve multiple requests concurrently. Pass 
--max-tasks=3
to the constructor of `http.Server`; see https://github.com/toitlang/pkg-http/blob/main/src/server.toit#L34.
(it could be that 3 isn't the perfect setting for you)
h
That helps, but unfortunately it postpones the problem. So the first three files download very fast and after that there's a 30 second delay. Increasing to 5 tasks leaves 1 file with a 30 second delay, increasing it to 6 tasks solves it. It seems a bit arbitrary, but I haven't looked at the source code though, there may be a good reason for that behaviour. Still, a 30 second delay...
k
Your browser may download multiple resources in parallel.
I think it is 6 for Chrome.
(connections per server)
Can't claim that I fully understand why you see the 30s delay though. Maybe @erikcorry knows?
h
Yes, the browser sending 6 requests at a time I can understand, but this 30 second delay... Ah, 30 seconds is the "--.read_timeout=DEFAULT_READ_TIMEOUT" parameter. And it is used in line 143 and further in server.toit: while true: request := null with_timeout read_timeout: request = connection.read_request
I can add a Duration parameter for a timeout of 3 seconds to the constructor of the http.Server and then the wait is 3 seconds.
e
The browser keeps trying to open connections instead of using the ones it already has open. After leaving a connection unused for 30s, it times out and gets closed, and at that point we can accept another connection.
I'm not sure what the best solution is here. Chrome is not expecting a server to have trouble with 6 parallel connecitons, but it's a lot of TCP connections for an ESP32.
Reducing the timeout to 3s would be an improvement, but it would be better if Chrome just reused the connection it already has instead of opening new ones.
h
I think it probably does. Safari also uses 6 connections AND sends a keep-alive header. I think 6 connections are used because for any ordinary website these days a lot of stuff needs to be downloaded. When I look at a regular news-site in the Netherlands I see 262 documents being downloaded. Not all from the same domain, but the main domain has 11 documents and the largest number of docs from one domain is +- 100!
There's room for improvement here on the part of the browsers I think. But people want fast loading websites, which is what you normally get when 6 connections are used by default.
e
It looks like Chrome only opens two connections for very simple websites.
If you add a few more assets it starts opening more.
For the example in pkg-dhs-simple-server (captive portal) it only opens two.
When I add more assets it opens six.
h
OK, I'll reduce my assets. I am not sure I can go as low as two :-). Thanks for your help @erikcorry !
e
That's not a great solution 😦
I'll check if accepting the socket, then immediately closing it causes the browser to use the ones it already ahs.
has
h
It isn't a great solution, but an esp32 is a restricted environment, I can live with that. I'll give it a go. svg's can be included in the html, css and js too (highly unrecommended security wise, but ...). Hopefully XHR requests don't count for opening a new connection - but I could use a websocket then which is a much better solution.
e
Immediately closing the socket causes broken-image icons in the browser. It doesn't just fall back to the connection it already has open.
h
Thanks for checking. That was fast.
k
We just need HTTP/2 support 🎅
e
Heh 🙂
For a server that does not use HTTPS, 30s is a very long timeout on unused connections. They can be set up much faster than that, so it's a bad tradeoff.
We should probably reduce that.
h
I've reduced the timeout to just --ms=10 and that works fine here with --max-tasks=1, so I'm good for now :-). Thanks guys.
e
That's agressive :-).
I'm still thinking about whether I can make a nicer fix here, but in the mean time we should use the latest HTTP library in the captive portal example https://github.com/toitlang/pkg-dns-simple-server/pull/9
PreviousNext
Powered by