Server Side Events (SSE) as short as possible

What makes Server Side Events different?

Usually, clients request data from the server and server responds with data.

Usual flow:

[client] -> [server]
[server] -> [client]

With server side events, you can send data from the server without the client making a request.

Unlike websockets, Server Side Events are one-way, only server can send data to client and not the other way around.

SSE in Go with langchaingo

It’s pretty simple to implement SSEs with langchaingo.

Create a handler:

	http.HandleFunc("/stream", func(w http.ResponseWriter, r *http.Request) {
		handleStream(w, r, llm)
	})

Here’s how the handler looks like:

func handleStream(w http.ResponseWriter, r *http.Request, llm llms.Model) {
  // ...
}

Inside the handler we need to set a few headers:

	// Set SSE headers
	w.Header().Set("Content-Type", "text/event-stream")
	w.Header().Set("Cache-Control", "no-cache")
	w.Header().Set("Connection", "keep-alive")
	w.Header().Set("Access-Control-Allow-Origin", "*")

And then just use the WithStreamingFunc method from langchain.

  prompt = "What would be a good company name for a company that makes colorful socks?"

	ctx := r.Context()
	_, err := llms.GenerateFromSinglePrompt(
		ctx,
		llm,
		prompt,
		llms.WithStreamingFunc(func(ctx context.Context, chunk []byte) error {
			fmt.Fprintf(w, "%s", string(chunk))
			if f, ok := w.(http.Flusher); ok {
				f.Flush()
			}
			return nil
		}),
	)

	if err != nil {
		fmt.Fprintf(w, "\ndata: ERROR: %v\n\n", err)
		return
	}

We write the bytes as they come to io.Writer, the f.Flush() makes sure bytes are written as soon as we get them.

You might want to follow me on X at @niyabits for a post on how to use SSEs on the frontend.