Project Logs

Collapse

• Full duplex

  lion mclionhead • 11/20/2025 at 10:10 • 0 comments

  Wrote a full duplex tester which is really good at causing packet collisions.  It quickly caused the phone to not receive a packet for over 1 second.  It could legitimately be a delay from a wifi collision.  

  The problem is compounded by the wrapping of all the  TCP packets inside another TCP connection inside wifi.  The only way it might work is by handling all the packets on the client & communicating with the phone in a simpler proprietary protocol, but TCP is already about as minimal as it can go.  At minimum, all the ACKs from Linux need to be discarded & all the ACKs from the phone need to be created on the client.  There's a lot of overhead from wrapping TCP inside TCP, but it might be bearable without all the ACKs.

  The ACK filtering got the full duplex test working quite well but SCP was still as bad as before & TCP still had a lot of resends.   It's expecting to use the window size to throttle its sending speed instead of the delaying of the ACKs.  The next step would be asynchronously writing to the phone while sending ACK to the client so the ACK isn't delayed.  That would require a full flow control algorithm using the window size or enough RAM to buffer the largest file it would send.

  That once again did nothing to speed up SCP but brought the number of resends down another notch.  Big websites typically don't have any more resends.  Only SCP has a small remaneing number of them.  It once again leaned towards artifacts of using multiple wifi hops or bugs in the Linux tun driver.

  With performance being good enough, it was time to wrap it up.
  

• Practical networking via proxy server

  lion mclionhead • 11/16/2025 at 01:03 • 0 comments

  Finally got full web browsing going on phone proxy.  It entailed deleting the default route on the phone's access point, toggling wifi on the phone to force it to use LTE.  Then run phoneproxy on the browser computer to direct it to the phone.
  

  Adding support for duplicate packets, sorting out of order packets, & setting PSH everywhere possible got it loading all the big sites through phone proxy & the data plan.  It takes a while to get started, but eventually goes at full 5G speeds.  Either it's not practical for the phone company to disable this kind of proxy server or what lions did was really hard.  They could always disable all forms of I/O except LTE, making phones even more crippled than they already are. Imagine having to disable data in order to access a camera.
  

  Setting the PSH flag was required to get useful latency in SSH, but was only possible from the phone to the PC.  It's not possible to force java.net.socket to set the PSH flag or get the PSH flag from java.net.socket. 
  

  It doesn't seem to be getting any out of order packets & the packet sorting doesn't support more than 4GB per connection.  Handling sequence number wraparounds is a big deal.  4GB file transfers on this data plan are unlikely.  Good old telnet was essential for testing.
  

  -------------------------------------------------------------------------------------------
  

  SCP over phoneproxy starts at 800kbyte/s, then gets slower & slower after a fresh install of the phone app until it eventually hits dialup speed or stalls.  It actually doesn't return to 800kbyte/s until the app is reinstalled.  Restarting the app briefly gets it to 200kbyte/s.
  

  Then the phone seems to close the phoneproxy socket after 2 minutes without activity.  Wireshark shows a lot of dup ACK & retransmissions from the client but no smoking gun when it stalls.  After cancelling, the phone sends out a lot of delayed ACKs which cause the client to send a lot of RST. 
  

  SCP writes data to the phone concurrently with payloads from the phone.  A smarter protocol would bundle payloads from the phone in the ACKs to the client, but payloads to the phone requiring an ACK from the phone seem to be extremely rare compared to payloads from the phone.
  

  There are multi second delays between some TCP sends from the client & the ACKs from the phone.  These cause resends after 200ms to 2 seconds.  The 200ms value is the TCP RTO & is not configurable.  The phone doesn't drop anything, but it needs the resends to kick the socket reads.  The resends are always client to phone.

  Padding the space between packets with 0xff until they hit 1024 bytes actually boosts the initial speeds to 800kbyte/s, but it always drops back to dialup.
  

  Setting TCP_NODELAY, setting all the PSH flags got higher speeds.  Nothing reduced the resends.

  -----------------------------------------------------------------------------------------------

  Large file HTTPS GETs from archive.org go smoothly with no resends & no need for padding.  Not padding the packets actually gets HTTPS up to 1MB/s.  Padding slows it down to 400kbyte/s.  Similar speeds happen with HTTP on the LAN.

  Full duplex is where the problems are happening.

• Virtual TCP

  lion mclionhead • 11/08/2025 at 19:35 • 0 comments

  The great challenge was testing.  It ended up using an offline raspberry pi as a test client & the phone's access point as a test server.

  On the test client, we have
  

  Kernel IP routing table
  Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
  0.0.0.0         10.12.0.1       0.0.0.0         UG    0      0        0 phone0
  10.0.5.0        10.0.10.11      255.255.255.0   UG    0      0        0 eth0
  10.0.10.0       0.0.0.0         255.255.255.0   U     0      0        0 eth0
  

  Phoneproxy's virtual ethernet is the 0.0.0.0 gatway.  Then, we have to access the phone itself with a 10.0.5.0/24 net.  Finally, ssh into the raspberry pi requires a 10.0.10.0/24 net.

  phone0: flags=4305  mtu 1500
          inet 10.12.0.1  netmask 255.255.255.255  destination 10.12.0.1
          inet6 fe80::284d:494d:ae58:cbec  prefixlen 64  scopeid 0x20
          unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)
  

  The virtual ethernet is a 10.12.0.1/32 net to make it go to the phone for all but 10.12.0.1.

  eth0: flags=4163  mtu 1500
          inet 10.0.10.11  netmask 255.255.255.0  broadcast 10.0.10.255
          inet6 fe80::dea6:32ff:fe94:a992  prefixlen 64  scopeid 0x20
          ether dc:a6:32:94:a9:92  txqueuelen 1000  (Ethernet)
          RX packets 8817854  bytes 1334993023 (1.2 GiB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 6952901  bytes 204012900 (194.5 MiB)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  
  eth0:1: flags=4163  mtu 1500
          inet 10.12.0.2  netmask 255.255.255.255  broadcast 10.255.255.255
          ether dc:a6:32:94:a9:92  txqueuelen 1000  (Ethernet)
  

  A test TCP server goes on the phone's access point 10.0.10.11.  It has 10.12.0.2.
  

  Another test TCP server goes on a container on the development PC as a reference.
  

  The test TCP server needs to do the basic tasks, open a connection, send from each side, close from each side, send large packets, stream multiple packets.

  ---------------------------------------------------------------------------------

  After the basic connection, transmit, receive, & close, the quality time with wireshark began.  It usually came to 1 corner case per day: failed DNS lookups, closing connections on the phone, packet fragmenting, multiple packets, cleanup after disconnecting the client.

  TCP is quite a beast.  Lions only ever implemented ICMP & UDP before.
  

  The mane problems were caused by timestamps.  Leaving out timestamps greatly simplified the algorithm.
  

  When sending synthetic packets to the client, the phone needs to track how full the client window is by subtracting the ACK sequence numbers from the sent sequence numbers.  Then it needs to wait for enough ACKs before sending.  Some implementations only send 1 packet per ACK.  The phone actually ended up too slow to fill the client window.
  

  The trick is testing out of order packets.  The packets are only in order when phoneproxy is running on the same machine as the browser.  It might be a reasonable requirement, since it's rarely going to be used & the phone accepts multiple connections.

• Virtual DNS

  lion mclionhead • 11/07/2025 at 08:50 • 0 comments

  DNS entailed handling A, AAAA, MX, & PTR requests.  The only real data it returns is the IPv4 address in the A request.  All the other bits are hard coded to dummy values.  The DNS server address is ignored.  It just handles all UDP sent to port 53.  Linux won't forward DNS requests to the phone unless the DNS server is outside the virtual ethernet device so lions simply use the global Comca$t server 75.75.75.75 as a dummy.

  root@truckcam:/root% host -v cnn.com
  Trying "cnn.com"
  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56858
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
  
  ;; QUESTION SECTION:
  ;cnn.com.                       IN      A
  
  ;; ANSWER SECTION:
  cnn.com.                3600    IN      A       151.101.67.5
  
  Received 41 bytes from 75.75.75.75#53 in 89 ms
  Trying "cnn.com"
  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19994
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
  
  ;; QUESTION SECTION:
  ;cnn.com.                       IN      AAAA
  
  ;; AUTHORITY SECTION:
  cnn.com.                3600    IN      SOA     x. x. 4294967295 86400 7200 2419200 3600
  
  Received 63 bytes from 75.75.75.75#53 in 0 ms
  Trying "cnn.com"
  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49886
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
  
  ;; QUESTION SECTION:
  ;cnn.com.                       IN      MX
  
  ;; ANSWER SECTION:
  cnn.com.                3600    IN      MX      10 x.
  
  Received 42 bytes from 75.75.75.75#53 in 9 ms
  

  root@truckcam:/root% host hackaday.com
  hackaday.com has address 192.0.66.96
  hackaday.com mail is handled by 10 x.
  
  
  root@truckcam:/root% host cnn.com
  cnn.com has address 151.101.67.5
  cnn.com mail is handled by 10 x.
  
  root@truckcam:/root% host cnn.com
  cnn.com has address 151.101.67.5
  cnn.com mail is handled by 10 x.
  
  root@truckcam:/root% ping heroinewarrior.com
  PING heroinewarrior.com (216.105.38.10) 56(84) bytes of data.
  64 bytes from x (216.105.38.10): icmp_seq=1 ttl=63 time=38.7 ms
  64 bytes from x (216.105.38.10): icmp_seq=2 ttl=63 time=248 ms
  64 bytes from x (216.105.38.10): icmp_seq=3 ttl=63 time=60.4 ms
  64 bytes from x (216.105.38.10): icmp_seq=4 ttl=63 time=86.8 ms
  64 bytes from x (216.105.38.10): icmp_seq=5 ttl=63 time=116 ms
  64 bytes from x (216.105.38.10): icmp_seq=6 ttl=63 time=206 ms
  64 bytes from x (216.105.38.10): icmp_seq=7 ttl=63 time=59.4 ms
  ^C
  --- heroinewarrior.com ping statistics ---
  7 packets transmitted, 7 received, 0% packet loss, time 6003ms
  rtt min/avg/max/mdev = 38.713/116.423/247.540/74.151 ms
  root@truckcam:/root%
  
  
  

  Some programs might malfunction if the reverse DNS name is always x & the authoritative name server is always x.  These bits could always be forwarded to the real DNS, with a lot of effort.  It might need a cache for reverse DNS.

• ICMP via proxy server

  lion mclionhead • 11/07/2025 at 00:46 • 0 comments

  Phone tethering in the past was possible by adjusting the TTL on the client, but this was closed long ago by creating a dedicated rmnet device just for tethering.

  Lions last used tethering on a daily basis while commuting, 6 years ago.

  In recent years, interest in tethering began to return as xfinity/comca$t reliability went downhill & data plans got ever cheaper.  Paid tethering was still proving very unreliable for some reason, compared to the apps on the phone.  It wasn't used often enough to justify the cost.  Simultaneously, modern phone plans leave a lot of data allowance unused.
  

  A big question was how to connect a client to the phone.  ADB & USB would be quite difficult & unreliable.  2.4Ghz wifi was quite slow & unreliable.  By 2024, 5Ghz wifi finally made it to cheap phones & drastically outperformed any other method.  It didn't seem to be throttled when LTE was the default gateway.  The kaboom case is if wifi is throttled when LTE is the default gateway, but a full proxy implementation would be the easiest way to test it.
  

  The idea is the phone & client would attach to an offline, raspberry pi access point.  Then the phone would use its LTE to access the great beyond.  The client would access the phone's wifi through a virtual ethernet device.  The raw packets from the client would be mangled on the phone to look like they came from a phone app.
  

  Helas, it's not possible to send raw packets to the rmnet devices on a cheap phone.  The phone app would have to analyze every raw packet from the client & create its own copy of every connection by using the Java library.  It would act like a virtual machine.

  Raw packet mangling would be possible with a full time server on the internet side of the phone, but it would cost more than a data plan.
  

  ---------------------------------------------------------------------------------------------------------------------------------------------------------
  

  The trick with testing is setting up a dedicated raspberry pi with its default route being the virtual ethernet device connected to the proxy server.  A test of translating raw ICMP packets from a client into InetAddress.isReachable commands on the phone showed promise.  Next would be making a DNS server that read raw UDP & called InetAddress.getByName

  The great task would be translating raw TCP packets into custom Socket commands on the phone.
  

View all 5 project logs