KevinSimper

Today I want to show how streaming an AI prompt response from a CloudFlare worker. We have been used to know that ChatGPT streams the response of the large language model, it is pretty mesmerizing and adds to the fealing of a thinking mashine that you can see the dialog as it is coming.

Simulating the response

We don't want to call an AI model during development just to simulate the LLM sending the events.

We can work with streams in Node.js by creating a TransformStream, that will allow us to both write and read the stream.

let { readable, writable } = new TransformStream();

When I started building a server for handling forward_auth requests, it was like stepping into uncharted territory. Googling this concept usually leads you to comprehensive but heavyweight solutions like Authelia or middleware examples in Traefik and Caddy. These are great for robust setups but overkill when you just need a simple, functional proof of concept.

Forward authentication is a mechanism where a reverse proxy delegates user authentication to an external server. The flow looks like this:

1. A request reaches the reverse proxy (e.g., Traefik, Caddy).
2. The proxy forwards the request to an authentication server for validation.
3. The authentication server responds with either:
   1. HTTP 2xx: User is authenticated, request proceeds to the backend.
   2. Otherwise: User should be redirected to the login page. All headers are taken from the auth service and returned

When developing software, decisions often come down to trade-offs between simplicity, readability, and maintainability. One recurring choice is whether to use a function or a class for a task. While both have their place, I often lean toward functions for many scenarios. Here’s why they can be a great option.

Why Choose Functions?

Fewer Side Effects

Functions tend to be straightforward: they take inputs, produce outputs, and avoid hidden states. This simplicity makes them easier to reason about and debug. If something breaks, you don’t have to untangle a web of interdependent attributes to locate the issue.

When testing, pure functions shine. They’re predictable: give them valid inputs, and they’ll always produce the same outputs. This predictability simplifies both writing and maintaining tests.

Coming from Node.js async code is the norm, and when async/await was introduced a big shift happened since it was competing with the callback approach where you could archive the same.

In Python it is not the same, Python is by default sync and webservers relies on WSGI (Web Server Gateway Interface) like gunicorn to handle multiple connections on a server.

So python tutorials does not normally talk or show async code examples, so learning it in python is defenitely harder.

The asyncio package

This package includes a lot of the functionality on how to control flow in your now async code. This blog shows really well the different methods like async.create_task, asyncio.wait, asyncio.gather and how to use them.

I have been creating a new Python project and when you start a project you always have the possibility to do things smarter than last time.

I have previous used pyenv or venv, but they never felt good for many reasons. The Python ecosystem feels more fragmented and powerful. For beginners it feels like the jump from simple dependency management to advance is a large gap.

https://docs.astral.sh/uv/

A new and simple package manager that scales

UV feels like the right package manager for beginners now. It takes many of the good things from the popular Cargo in Rust and brings it to Python.