https://toitlang.org/ logo
Join Discord
Powered by
Samsung phones and captive portal
# help
e
Here to discuss the troubles with Samsung phones
Samsug phones send a series of very strange DNS queries in order to determine whether they are talking to a real DNS server or being spoofed.
Rikke was kind enough to send me a dump of the DNS queries.
@Rikke
Among other things it looks up 
google.com.onion
and 
www.goooooooooooooooooooooooooooooooooooooooooooooooooooooooooogle.com
r
I can provide the DNS query from iphone, oneplus and huawei too, but not sure if it will help.
k
That's what I always do manually!
e
It looks to be related to this patent that I found: https://uspto.report/patent/app/20210258279
To be honest this looks like a really dumb idea. They are using machine learning to evaluate the results of probing the DNS server with strange requests.
r
I can see the 
connectivitycheck.gstatic.com
and this should work 🤔
e
The packet causing the exception looks wrong. I can't understand how it would be parseable.
So we could throw a different exception, but it would still be an exception.
Do you have to restart the device after the exception is thrown?
Copy code
OUT_OF_BOUNDS
  0: ByteArrayBase_.to_string  <sdk>/core/collections.toit:1403:5
  1: parts_                    <sdk>/net/modules/dns.toit:356:16
  2: decode_name               <sdk>/net/modules/dns.toit:346:3
  3: SimpleDnsServer.lookup.<block>.<block> ../src/pkg_dns_simple_server/src/dns_simple_server.toit:90:19
  4: SmallInteger_.repeat      <sdk>/core/numbers.toit:1194:3
  5: SimpleDnsServer.lookup.<block> ../src/pkg_dns_simple_server/src/dns_simple_server.toit:89:18
  6: catch.<block>             <sdk>/core/exceptions.toit:124:10
  7: catch                     <sdk>/core/exceptions.toit:122:1
  8: catch                     <sdk>/core/exceptions.toit:73:10
  9: SimpleDnsServer.lookup    ../src/pkg_dns_simple_server/src/dns_simple_server.toit:34:18
 10: run_dns                   ../src/pkg_dns_simple_server/examples/captive_portal.toit:308:23
 11: run_portal.<lambda>       ../src/pkg_dns_simple_server/examples/captive_portal.toit:146:13
r
No the esp continues. The textfile was a few seconds of samsung spam, and it got 2 exceptions in that time
e
Ah yes, there's a 
catch --trace
in 
SimpleDnsServer.lookup
so it keeps going.
r
I can see
Copy code
DEBUG: client connected {peer: 192.168.4.2:53410}
DEBUG: incoming request {peer: 192.168.4.2:53410, path: /gen_204}
which should be redirected, because it is in the temporary redirects
Copy code
TEMPORARY_REDIRECTS ::= {
  "generate_204": "/",    // Used by Android captive portal detection.
  "gen_204": "/",         // Used by Android captive portal detection.
  // Add more redirects in order to create an alias for a file.
}
so maybe this dns spoofing thing is ruining it
e
Those two things should be independent.
It could be that the Samsung phone is not understanding the redirection.
The iPhones don't like. They just want the portal page to be served up under the name they use 
hotspot-detect.html
rather than being redirected.
r
My huawei also tries generate_204 and gen_204 and it works, so it must be a samsung thing. But hopefully not related to the spoofing thing
e
I've added a remove_host method to the DNS server so you can give it domains that should not get the default IP if there is a query for them.
But I think it's probably not needed here.
I think if the Samsung phone is doing the HTTP request for gen_204 then it got past the DNS problems.
So the question is why it is not responding to the answer of that request by popping up the login page.
Could be that it needs the same workaround as the iPhone with the "silent redirect" that we have for 
hotspot-detect.html
.
Worth trying.
r
I will try it out tomorrow 🙂
Yeah it did not make cp pop up on Samsung. But this is what the phone says when its connecting to cp and spamming dns stuff. I will see if i can find a hidden request so i can redirect
I do have access to an old samsung at home. I can try and see if it has the same problem
Okay I've tried to redirect from different requests, but with no luck. I fear the gen_204 does not give the response it needs to detect the captive portal 😕 I will see what my old samsung does when I get home, and see if it reveals the magic request
e
I released version 0.3 of the DNS simple server with that small change, so that you can respond with an unknown domain error when it asks about domains that don't exist. I suspect that you should return an error for the google.com.onion domain, and perhaps also for the gooooooooooooooooooooooooogle domain.
r
I will try that
If I call remove host on these weird websites, it should give an error on serial log? This is how I use it: run_dns is started as a task and given ip from captive portal
Copy code
run_dns network/net.Interface:
  my_ip := network.address

  socket := network.udp_open --port=53

  hosts := SimpleDnsServer my_ip  // Answer my IP to all queries.
  hosts.remove_host "chat-e2ee-mini.facebook.com"
  hosts.remove_host "mqtt-mini.facebook.com"
  hosts.remove_host "www.goooooooooooooooooooooooooooooooooooooooooooooooooooooooooogle.com"
  hosts.remove_host "google.com.onion"
  hosts.remove_host "graph.facebook.com"  
  hosts.remove_host "chat-e2ee-mini.facebook.com"   
  while true:
    datagram /udp.Datagram := socket.receive
    response := hosts.lookup datagram.data
    if response:
      e := catch:
        socket.send
          udp.Datagram response datagram.address
      if e:
        //print "udp exception: $e"
I don't see any errors
Copy code
Decode name:
  #[0x00, 0x11, 0x01, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x67, 0x6f, 0x6f]
                                                                            ^^^^
  #[0x67, 0x6c, 0x65, 0x03, 0x63, 0x6f, 0x6d, 0x05, 0x6f, 0x6e, 0x69, 0x6f, 0x6e, 0x00, 0x00, 0x01]
  #[0x00, 0x01]
  Got google.com.onion
Decode name:
  #[0x00, 0x11, 0x01, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x67, 0x6f, 0x6f]
                                                                            ^^^^
  #[0x67, 0x6c, 0x65, 0x03, 0x63, 0x6f, 0x6d, 0x05, 0x6f, 0x6e, 0x69, 0x6f, 0x6e, 0x00, 0x00, 0x01]
  #[0x00, 0x01]
  Got google.com.onion
But im no dns wizard
e
It doesn't print anything, just returns a DNS packet that tells the phone the domain does not exist.
Copy code
if response_address == null:
          return response.create_error_ dns.ERROR_NAME
You could put a 
print_
here.
The long www.goooooooooooooooooooooooooooooooooooooooooooooooooooooooooogle.com actually exists though!
Copy code
$ ping www.goooooooooooooooooooooooooooooooooooooooooooooooooooooooooogle.com
PING goooooooooooooooooooooooooooooooooooooooooooooooooooooooooogle.com (34.102.136.180) 56(84) bytes of data.
64 bytes from 180.136.102.34.bc.googleusercontent.com (34.102.136.180): icmp_seq=1 ttl=54 time=49.8 ms
r
Okay .onion does not exist 😂 and by printing I can verify it works, thanks! I do still get the exception. It looks like each query is printed twice, and the first one sometimes fails.
Copy code
Decode name:
----
Received a Toit system message. Executing the command below will
make it human readable:
----
  #[0x07, 0x0e, 0x01, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x61, 0x6c, 0x74]

***
EXCEPTION error. 
OUT_OF_BOUNDS
  0: ByteArrayBase_.to_string  <sdk>/core/collections.toit:1403:5
  1: parts_                    <sdk>/net/modules/dns.toit:356:16
  2: decode_name               <sdk>/net/modules/dns.toit:346:3
  3: SimpleDnsServer.lookup.<block>.<block> ../src/pkg_dns_simple_server/src/dns_simple_server.toit:105:19
  4: SmallInteger_.repeat      <sdk>/core/numbers.toit:1194:3
  5: SimpleDnsServer.lookup.<block> ../src/pkg_dns_simple_server/src/dns_simple_server.toit:104:18
  6: catch.<block>             <sdk>/core/exceptions.toit:124:10
  7: catch                     <sdk>/core/exceptions.toit:122:1
  8: catch                     <sdk>/core/exceptions.toit:73:10
  9: SimpleDnsServer.lookup    ../src/pkg_dns_simple_server/src/dns_simple_server.toit:51:18
 10: run_dns                   ../src/pkg_dns_simple_server/examples/captive_portal.toit:288:23
 11: run_portal.<lambda>       ../src/pkg_dns_simple_server/examples/captive_portal.toit:148:13
***

                                                                            ^^^^
  #[0x33, 0x2d, 0x6d, 0x74, 0x61, 0x6c, 0x6b, 0x06, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x03, 0x63]
  #[0x6f, 0x6d, 0x00, 0x00, 0x01, 0x00, 0x01]
  Got alt3-mtalk.google.com
Decode name:
  #[0x07, 0x0e, 0x01, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x61, 0x6c, 0x74]
                                                                            ^^^^
  #[0x33, 0x2d, 0x6d, 0x74, 0x61, 0x6c, 0x6b, 0x06, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x03, 0x63]
  #[0x6f, 0x6d, 0x00, 0x00, 0x01, 0x00, 0x01]
  Got alt3-mtalk.google.com
The 2nd "Decode name:" is where the print from remove_hosts is being printed:
Copy code
Decode name:
  #[0x58, 0x16, 0x01, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x09, 0x6d, 0x71, 0x74]
                                                                            ^^^^
  #[0x74, 0x2d, 0x6d, 0x69, 0x6e, 0x69, 0x08, 0x66, 0x61, 0x63, 0x65, 0x62, 0x6f, 0x6f, 0x6b, 0x03]
  #[0x63, 0x6f, 0x6d, 0x00, 0x00, 0x01, 0x00, 0x01]
  Got mqtt-mini.facebook.com
Decode name:
  #[0x58, 0x16, 0x01, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x09, 0x6d, 0x71, 0x74]
                                                                            ^^^^
  #[0x74, 0x2d, 0x6d, 0x69, 0x6e, 0x69, 0x08, 0x66, 0x61, 0x63, 0x65, 0x62, 0x6f, 0x6f, 0x6b, 0x03]
  #[0x63, 0x6f, 0x6d, 0x00, 0x00, 0x01, 0x00, 0x01]
  Got mqtt-mini.facebook.com
NO I DONT LIKE
It looks like something is not cleared, here's two(of the same) google.com where the last one is different from first one. The next two(of the same) will have the first one fail. I dont know if this helps with debugging, but I attached 2 files with dns query prints in 
decode_name
around the exception
conversationconversation
e
The exception should be harmless. because it is caught and the server keeps going.
In the next release of Toit it will be more informative, but there will still be an exception.
As far as I can tell the DNS packet we get is corrupt so there's not a lot we can do.
r
Okay 👍
e
@bmentink is having similar problems with Motorola.
I wonder if it's a general Android problem.
Last I checked it worked fine with my Google phone.
r
Samsung s4 mini does not automatically open CP, even though it tries gen_204 and generate_204. OnePlus 6 works as expected. Huawei p 2019 smart asks if I want to open a login page after connecting to wifi hotspot iphone 8 works with the 
/hotspot-detect.html
redirect to index.html Samsung S21+ does not automatically open CP, even though it tries gen_204 and generate_204. But I can open it myself by entering ip 192.168.4.1 I've looked online for ways to automatically open CP on all phones, but it seems like its a general problem. Each phone brand behaves differently, so its not even a question of android or iphone 😦
I think i stole one of our salesmen's new iphone, and it worked as expected. Most likely because of the redirect
Okay I hijacked someone with a motorola g62 5g and it worked. first time connecting to the hotspot, it took like 20 sec to open it automatically. the 2nd time it opened instantly
The huawei can consistently produce this exception when connecting, but it works as expected really. The esp does not restart because they catch the exception
Copy code
**
EXCEPTION error. 
Connection closed
  0: tcp_write_                <sdk>/net/modules/tcp.toit:199:3
  1: TcpSocket.write           <sdk>/net/modules/tcp.toit:165:16
  2: Writer.write              <sdk>/writer.toit:39:23
  3: ChunkedWriter.write_header_ <pkg:pkg-http>/chunked.toit:92:13
  4: ChunkedWriter.write       <pkg:pkg-http>/chunked.toit:72:5
  5: ResponseWriter_.write     <pkg:pkg-http>/server.toit:124:18
  6: handle.<block>            ../src/pkg_dns_simple_server/examples/captive_portal.toit:264:12
  7: Map.get.<block>           <sdk>/core/collections.toit:2699:33
  8: HashedInsertionOrderedCollection_.find_body_ <sdk>/core/collections.toit:2176:3
  9: HashedInsertionOrderedCollection_.find_ <sdk>/core/collections.toit:2155:12
 10: Map.get                   <sdk>/core/collections.toit:2699:15
 11: Map.get                   <sdk>/core/collections.toit:2679:12
 12: handle                    ../src/pkg_dns_simple_server/examples/captive_portal.toit:261:26
 13: run_portal.<lambda>.<lambda> ../src/pkg_dns_simple_server/examples/captive_portal.toit:159:13
 14: Server.run_connection_.<block> <pkg:pkg-http>/server.toit:93:17
 15: catch.<block>             <sdk>/core/exceptions.toit:124:10
 16: catch                     <sdk>/core/exceptions.toit:122:1
 17: catch                     <sdk>/core/exceptions.toit:97:10
 18: Server.run_connection_    <pkg:pkg-http>/server.toit:92:7
 19: Server.listen.<lambda>.<block>.<block> <pkg:pkg-http>/server.toit:67:24
 20: catch.<block>             <sdk>/core/exceptions.toit:124:10
 21: catch                     <sdk>/core/exceptions.toit:122:1
 22: catch                     <sdk>/core/exceptions.toit:73:10
 23: Server.listen.<lambda>.<block> <pkg:pkg-http>/server.toit:66:16
 24: Server.listen.<lambda>    <pkg:pkg-http>/server.toit:53:24
**
e
Every phone producer that customizes Android makes it worse 😕
r
Yup 🙈
e
Happy to report that my Google Pixel still goes to the portal immediately.
I live in such a bubble that I don't know anyone who doesn't have an iPhone or a Pixel.
I took a look at how other do it, and there's an interesting list here: https://github.com/23ewrdtf/Captive-Portal/blob/master/dnsmasq.conf
Perhaps you should try adding all these to the DNS server with the own address, and returning an error for all others (not having a default IP).
I'm not sure why they have the correct IP address for each of those - feels like that's not useful on a captive portal that can't route to those correct addresses.
If people's phones are on Android 11 by now (it's from 2020) then we could use this technique : https://developer.android.com/about/versions/11/features/captive-portal
Something like this.
Copy code
diff --git a/examples/captive_portal.toit b/examples/captive_portal.toit
index 418bab8..d8320b2 100644
--- a/examples/captive_portal.toit
+++ b/examples/captive_portal.toit
@@ -23,6 +23,17 @@ import .website.resources
 CAPTIVE_PORTAL_SSID     ::= "mywifi"
 CAPTIVE_PORTAL_PASSWORD ::= "12345678"
 
+PORTAL_PROBES ::= [
+    "hotspot.localnet",
+    "connectivitycheck.gstatic.com",
+    "www.gstatic.com",
+    "www.apple.com",
+    "captive.apple.com",
+    "clients3.google.com",
+    "www.msftconnecttest.com",
+    "connectivitycheck.platform.hicloud.com",
+]
+
 // On the device we use port 80 for the web server, but that is not
 // available to non-privileged users on desktop systems, so we fall
 // back on this port when testing on desktop Toit.
@@ -123,7 +134,10 @@ run_dns network/net.Interface:
 
   socket := network.udp_open --port=53
 
-  hosts := SimpleDnsServer my_ip  // Answer my IP to all queries.
+  hosts := SimpleDnsServer  // DNS server with no default answer.
+
+  PORTAL_PROBES.do:
+    hosts.add_host it my_ip  // Answer my IP to the known probe domains.
 
   while true:
     datagram /udp.Datagram := socket.receive
This still works with my Pixel, perhaps it's better with Samsung.
This says you have to use a specific IP range (200.200.'200.0/24) to persuade Samsung phones. Something we can also look into. Putting the link here so I don't forget it. https://stackoverflow.com/questions/59588154/captive-portal-for-android-phone-did-not-pop-up/59643642#59643642
r
Only samsung is a problem (and thats what i use now yay 🤡 ). I will try and add these probe sites, and see if I can trick samsung. After that I will see if I can set the dns server ip to 200.200.200.1 and DHCP server range to 200.200.200.10 - 200.200.200.250. 🧐
Maybe i dont need to set the dns server ip
e
Yes I think if you set the address to 200.200.200.0 with mask 255.255.255.0 then it automatically sets the DNS server's IP.
I don't see how to do that right now though.
r
Adding the 
PORTAL_PROBES
did not make the CP pop up samsung 😕
e
You removed the default too?
r
yes remove the "my_ip" part and add all the hosts from PORTAL_PROBES
e
We don't have a way to configure the IP address of the soft AP at the moment. This patch will permanently move it to 200.200.200.0/24
Copy code
diff --git a/src/resources/wifi_esp32.cc b/src/resources/wifi_esp32.cc
index 11fd104b..b6e629f3 100644
--- a/src/resources/wifi_esp32.cc
+++ b/src/resources/wifi_esp32.cc
@@ -384,7 +384,14 @@ PRIMITIVE(init) {
   // because they do not correctly check for malloc failure.
   esp_netif_t* netif = null;
   if (ap) {
+    esp_netif_ip_info_t two_hundred_network;
+    two_hundred_network.ip.addr = ESP_IP4TOADDR( 200, 200, 200, 1);
+    two_hundred_network.gw.addr = ESP_IP4TOADDR( 200, 200, 200, 1);
+    two_hundred_network.netmask.addr = ESP_IP4TOADDR( 255, 255, 255, 0);
+    esp_netif_inherent_config_t netif_config = ESP_NETIF_INHERENT_DEFAULT_WIFI_AP();
+    netif_config.ip_info = &two_hundred_network;
     esp_netif_config_t netif_ap_config = ESP_NETIF_DEFAULT_WIFI_AP();
+    netif_ap_config.base = &netif_config;
     netif = esp_netif_new(&netif_ap_config);
   } else {
     esp_netif_config_t netif_sta_config = ESP_NETIF_DEFAULT_WIFI_STA();
r
This gave me the prompt "log into wifi" 🧐 so this is what it needs
Hmm but does not allow me to href because its using the samsung captive portal app/thing for site 
connectivitycheck.gstatic.com
. This should be fixable in the html
e
Did you change the IP address in the DNS server to match (200.200.200.1)?
r
I just applied your patch
b
Does the above patch fix the issues? If so will it go into a following release?
r
it fixes the issue with captive portal not being detected on samsung phones, but its only tested on samsung galazy s22+
I tested on: Ubuntu 22.04 PC Samsung Galaxy S22+ Huawei P 2019 Smart IPhone 8 Motorola G62 5G
and yes, it changed the captive portal site from 192.168.4.1 to 200.200.200.1
b
@Rikke how can I test this on my Motorola phone, I don't have a build setup..
r
The motorola i tested on, worked fine without this patch. but I just apply the patch in the toit git repo and rebuild the sdk. this is the script i use
Copy code
#! /bin/bash

cd toit/

export IDF_PATH=$PWD/third_party/esp-idf
$IDF_PATH/install.sh
source $IDF_PATH/export.sh
make all
make esp32
Have you tested on other devices than your motorola?
b
Yep, all Apple devices seem fine..
Also macos MacBook air m1 works ok
My Motorola phone that didn't work is a G9 play
r
Its worth a try with the patch then
b
Once you have built the sdk how do you get Artemis to use it? I haven't attempted a build of toit yet
r
I dont use artemis so I dont know
b
Oh, ok to hard for me then, I will wait to see if it goes into a release..
r
You can always navigate to 192.168.4.1 if it does not automatically pop up for your device
b
Can't have the customer do that..😃
r
It actually worked fine with our technicians who installed our equipment
But its nicer when it works on all devices, but captive portal pop-up isn't that straightforward. A few minutes of research and you discover each phone brand decides to do things differently 🙂
f
Next release of Artemis should have support for custom firmware builds.
r
Looks like this captive portal with dns change does not work on windows 11 pc 🤡 i will investigate
redirects, like with iphone, works for windows as well.
Copy code
// WINDOWS PC works on 10 Pro and 11
  else if path == "/connecttest.txt": path = "index.html" // to fix windows pc captive problem
  else if path == "/redirect": path = "index.html" // to fix windows pc captive problem
It looks like it tries /connecttest.txt first and gets redirected to /redirect, but i had to redirect /redirect to open the site. two other possible checking sites are 
/canonical.html
and 
/success.txt?ipv4
but i did not need them for win 10 pro and win 11
e
Thanks I'll add that too.
b
Can you give the complete handling for 
handle_http_request
. i.e post the code please ..
I curently have the following in setup.toit:
Copy code
handle_http_request request/http.Request writer/http.ResponseWriter access_points/List -> Map?:
  query := url.QueryString.parse request.path
  resource := query.resource
  if resource == "/": resource = "index.html"
  if resource == "/hotspot-detect.html": resource = "index.html"  // Needed for iPhones.
  if resource.starts_with "/": resource = resource[1..]
r
Copy code
handle request/http.Request writer/http.ResponseWriter -> none:
  path := request.path
  if path == "/": path = "index.html" 
  else if path == "/hotspot-detect.html": path = "index.html"
  else if path == "/connecttest.txt": path = "index.html"
  else if path == "/redirect": path = "index.html"
b
I am having lot's of trouble connecting using captive portal ever since to artemis > 0.9.0 or when CP used 200.200.200.1 ip's , can someone please look at the following log to see any clues ..thanks https://cdn.discordapp.com/attachments/1118162779069763696/1147657388208431164/log.txt
@erikcorry or @Rikke
f
I have the experience that the routers don't like too much when a device establishes connections frequently. Could that be a reason for your issues?
b
This is my latest attempt. This time an out of memery error, I have no idea what is going on. Getting very frustrated as I am mean't to be shipping this product soon .. https://cdn.discordapp.com/attachments/1118162779069763696/1148363887452901416/log2.txt
Is there the latest version of artemis-zygote somewhere I can check against my code?
k
@bmentink It looks pretty weird that you're getting out-of-memory when you have lots of free memory. Also, the SSID looks a bit iffy with the 
+
signs in there. Are they typed in as spaces?
The https://libs.toit.io/encoding/url/library-summary#decode(1%2C0%2C0%2C) method doesn't change 
+
into
, so maybe that is causing you some issues here. @erikcorry probably knows the rationale for this, but maybe the 
QueryString
should be extended with a convenient way to do fix the 
+
signs as part of decoding a query?
b
Hi @kasperl yes that particular SSID had spaces, which were converted to + , maybe we should support this?
k
@bmentink Fix is submitted for review 🙂
e
Getting an OOM when there is 48k free could be one of two things: 1) We reserve 160k for malloc, and the rest can be used for either malloc or Toit heap. If you use much less than expected for malloc then you could get an out-of-memory despite having free memory (that is malloc-only). Certain usage patterns could also cause this if you have a lot of mallocs that are in the common area instead of the malloc-only area, I haven't seen that happen, but it's theoretically possible. 2) One of your programs is trying to make a single very large allocation, 48k+.
In case it's 1) we will try a smaller malloc-only area in the next release: https://github.com/toitlang/toit/pull/1794
It looks like the device was power cycled in this trace
Copy code
[wifi] DEBUG: connected
[wifi] INFO: network address statically assigned {ip: 200.200.200.1}
[wifi] INFO: dns server address not supplied by network; using fallback dns servers
ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
k
@bmentink Artemis v0.9.7 is out and it comes with a fix for the 
QueryString.parse
issues with 
+
.
PreviousNext
Powered by