Skip to content

Instantly share code, notes, and snippets.

@anecdata
Last active November 22, 2026 02:47
Show Gist options
  • Select an option

  • Save anecdata/89b4458fd7c0c01c7fc0a9ff4be5b4e6 to your computer and use it in GitHub Desktop.

Select an option

Save anecdata/89b4458fd7c0c01c7fc0a9ff4be5b4e6 to your computer and use it in GitHub Desktop.
CircuitPython asyncio TCP multi-server multi-client
import time
import random
import os
import wifi
import socketpool
import asyncio
NUM_CLIENTS = 4
NUM_SERVERS = 2
BASE_PORT = 5000
MAXBUF = 64
class Socket:
""" https://github.com/adafruit/circuitpython/pull/7173 """
def __init__(self, s):
self.s = s
s.setblocking(False)
async def recv_into(self, buf):
await asyncio.core._io_queue.queue_read(self.s)
return self.s.recv_into(buf)
async def send(self, buf):
await asyncio.core._io_queue.queue_write(self.s)
return self.s.send(buf)
async def accepted_conn(server_num, conn, event):
"""handles a single server connection from a client"""
inbuf = bytearray(MAXBUF)
sconn = Socket(conn)
size = False
while not size:
try:
size = await(sconn.recv_into(inbuf)) # OSError: [Errno 128] ENOTCONN
except OSError as ex:
await asyncio.sleep(0)
print(f"{time.monotonic():.1f}s SERVER {server_num} received {size} bytes {inbuf[:size]}")
outbuf = f"{time.monotonic():.1f}s Hello client from server {server_num}".encode()
await(sconn.send(outbuf))
print(f"{time.monotonic():.1f}s SERVER {server_num} sent {len(outbuf)} bytes {outbuf}")
# sconn.close() # AttributeError: 'Socket' object has no attribute 'close'
conn.close()
event.set()
async def server(server_num):
"""persistent TCP server"""
BACKLOG = NUM_CLIENTS
s = pool.socket(pool.AF_INET, pool.SOCK_STREAM)
s.setsockopt(pool.SOL_SOCKET, pool.SO_REUSEADDR, 1) #
port = BASE_PORT + server_num
s.bind((host, port))
s.listen(BACKLOG)
print(f"{time.monotonic():.1f}s SERVER {server_num} listening on port {port}")
total = 0
while True:
print(f"{time.monotonic():.1f}s SERVER {server_num} accepting connections")
s.setblocking(False)
addr = False
while not addr:
try:
conn, addr = s.accept() # OSError: [Errno 11] EAGAIN
print(f"{time.monotonic():.1f}s SERVER {server_num} accepted connection from {addr}")
event = asyncio.Event()
t = asyncio.create_task(accepted_conn(server_num, conn, event))
await event.wait()
t.cancel()
total += 1
print(f"{time.monotonic():.1f}s SERVER {server_num} has processed {total} total connections")
except OSError as ex:
await asyncio.sleep(0)
await asyncio.sleep(0)
async def connected_client(client_num, s, event):
"""handles a single client connection to a server"""
inbuf = bytearray(MAXBUF)
ss = Socket(s)
outbuf = f"{time.monotonic():.1f}s Hello server from client {client_num}".encode()
size = await(ss.send(outbuf))
print(f"{time.monotonic():.1f}s CLIENT {client_num} sent {size} bytes {outbuf}")
size = False
while not size:
try:
size = await(ss.recv_into(inbuf)) # OSError: [Errno 11] EAGAIN
except OSError as ex:
await asyncio.sleep(0)
print(f"{time.monotonic():.1f}s CLIENT {client_num} received {size} bytes {inbuf[:size]}")
# ss.close() # AttributeError: 'Socket' object has no attribute 'close'
event.set()
async def client(client_num):
"""TCP client wrapper"""
total = 0
while True:
with pool.socket(pool.AF_INET, pool.SOCK_STREAM) as s:
port = random.randint(BASE_PORT, BASE_PORT + NUM_SERVERS)
print(f"{time.monotonic():.1f}s CLIENT {client_num} connecting to ({host}, {port})")
try:
s.connect((host, port)) # OSError: [Errno 119] EINPROGRESS
print(f"{time.monotonic():.1f}s CLIENT {client_num} connected to ({host}, {port})")
event = asyncio.Event()
t = asyncio.create_task(connected_client(client_num, s, event))
await event.wait()
t.cancel
total += 1
print(f"{time.monotonic():.1f}s CLIENT {client_num} has processed {total} total connections")
except OSError as ex:
await asyncio.sleep(0)
await asyncio.sleep(random.random())
async def main():
print(f"{time.monotonic():.1f}s creating {NUM_SERVERS} servers, {NUM_CLIENTS} clients")
tasks = []
for server_num in range(NUM_SERVERS):
tasks.append(asyncio.create_task(server(server_num)))
for client_num in range(NUM_CLIENTS):
tasks.append(asyncio.create_task(client(client_num)))
print(f"{time.monotonic():.1f}s {len(tasks)} tasks created")
while True:
await asyncio.sleep(0)
time.sleep(3) # wait for serial after reset
wifi.radio.connect(os.getenv('WIFI_SSID'), os.getenv('WIFI_PASSWORD'))
host = str(wifi.radio.ipv4_address)
pool = socketpool.SocketPool(wifi.radio)
asyncio.run(main())
@anecdata

anecdata commented Aug 25, 2025 •

Copy link
Copy Markdown
Author

A few key things are as-yet unimplemented here and in the CircuitPython core / asyncio library, and it eventually hard faults into safe mode (Hard fault: memory access or instruction error.). I'm sure there are more graceful ways of doing much of this.

@anecdata

Copy link
Copy Markdown
Author

Tested on Adafruit CircuitPython 9.2.8 on 2025-05-28; Adafruit QT Py ESP32-S3 4MB Flash 2MB PSRAM with ESP32S3

ESP32-S3 has 8 sockets. Each server uses 2 sockets. Each client uses 1 socket.

@anecdata

anecdata commented Sep 1, 2025 •

Copy link
Copy Markdown
Author

Hard fault was due to using 9.2.8, which still had this issue that had been largely attributed to socket servers. addendum: The minimal example was fixed by 10.0.0-beta.3, but the full code still hard faults... back to the drawing board.

Note that in CP 10 (10.0.0-beta.3), there are now only 4 TCP sockets available on espressif, rather than 8:

import time, wifi, socketpool
pool = socketpool.SocketPool(wifi.radio)
socks = []
while True:
    socks.append(pool.socket(pool.AF_INET, pool.SOCK_STREAM))
    print(f"{len(socks)}")  # 8 in CP 9; 4 in CP 10

Update: back to 8 sockets with adafruit/circuitpython#10609.

@anecdata

anecdata commented Feb 22, 2026 •

Copy link
Copy Markdown
Author

Retested with Adafruit CircuitPython 10.1.0-beta.1 on 2025-11-06; Adafruit QT Py ESP32-S3 4MB Flash 2MB PSRAM with ESP32S3 and issue reported here: adafruit/circuitpython#10775

@b-spatz

b-spatz commented Nov 3, 2026

Copy link
Copy Markdown

Any updates on resilience? I am looking for a robust CP pattern to monitor a TCP port and multiple UDP ports (the former for a lightweight web status service via HTTP).

@anecdata

anecdata commented Nov 3, 2026 •

Copy link
Copy Markdown
Author

I was just re-testing this with a recent CircuitPython 11.0.0-alpha.2 build, 3.5 hours and still running. The issue cites up to 8 hours before hardfault, so may depend on your tolerance for detecting and auto-restarting via safemode.py. I'll report back here when (if) it hardfaults.

For reasons stated above and in the issue, this does not seem to be standard CPython, and I'm not sure it's really supported, I had found it in a PR and just wanted to try it out. Depending on frequency of packets and packet size, you may be able to use normal non-async TCP sockets with appropriate timeouts.

But I'm not sure how to accomplish multiple UDP servers on a single CircuitPython device. I think you would need core support for async UDP with buffering. I didn't pursue that with this approach beyond the initial string of exceptions I encountered.

@anecdata

anecdata commented Nov 3, 2026

Copy link
Copy Markdown
Author

Still running after about 17.5 hours, but need to repurpose the hardware. I'm not convinced something was magically fixed, but this frequency of hardfaults (assuming it will eventually hardfault) may be tolerable.

@anecdata

anecdata commented Nov 3, 2026

Copy link
Copy Markdown
Author

You may want to comment on issue adafruit/circuitpython#10775 to ask about support for the pattern or other related concerns.

@b-spatz

b-spatz commented Nov 22, 2026 •

Copy link
Copy Markdown

Under a test harness offering the UDP traffic I'm monitoring, and the TCP interaction (web status server) every 1-5 seconds, I see variation in performance between boards with low RAM (e.g. Lilygo T-Dongle-S3 512KB SRAM only) and say 2MB PSRAM (generic ESP32-S3 "supermini" or Waveshare Zero, etc.). Although I've explicitly set non-blocking, I can elicit a hang with socket.send() (TCP) reliably on the low RAM boards despite all efforts to catch/handle errors, split-up the reply into smaller send()s, etc. FYI.

I agree: at this point it's unclear what is actually supported in Circuitpython, and especially with asyncio patterns -- which I now I am avoiding with hand-coded polling, etc. If I wanted that I'd be using C and not Circuitpython. Beggars can't be choosers I guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment