Implementing WebTransport over HTTP/2 using JavaScript in the browser

[guest271314]

I read this https://hpbn.co/http2/#streams-messages-and-frames which lead me here.

I'm scraping the surface of implementing a WebTransport server over HTTP/2 in the browser using WICG Direct Sockets TCPServerSocket. I've implemented a HTTP and WebSocket server already.

I'm kind of stuck at the beginning.

Any basic roadmap will be appreciated.

[HoneyryderChuck]

Hi ,

You can find examples on how to use this gem in the examples dir. I'm not familiar with webtransport nor with what you want to achieve to give you a better recommendation.

[guest271314]

I'm not familiar with Ruby. The linked article lead me here.

[guest271314]

I was able to start with the above linked article and eventually gather the basic header exachanges so I could write this is runtime agnostic JavaScript

              if (/^(POST|query)/i.test(request)) {
                const [body] = request.match(
                  /(?<=\r\n\r\n)[a-zA-Z\d\s\r\n-:;=]+/igm,
                );
                console.log({
                  body,
                });
                await writer.write(encode("HTTP/1.1 200 OK\r\n"));
                await writer.write(
                  encode("Content-Type: application/octet-stream\r\n"),
                );
                await writer.write(
                  encode("Access-Control-Allow-Origin: *\r\n"),
                );
                await writer.write(
                  encode("Access-Control-Allow-Private-Network: true\r\n"),
                );
                await writer.write(
                  encode(
                    "Access-Control-Allow-Headers: Access-Control-Request-Private-Network\r\n",
                  ),
                );
                await writer.write(encode("Cache-Control: no-cache\r\n"));
                await writer.write(encode("Connection: close\r\n"));
                await writer.write(
                  encode("Transfer-Encoding: chunked\r\n\r\n"),
                );

                const chunk = encode(body.toUpperCase());
                const size = chunk.buffer.byteLength.toString(16);
                await writer.write(encode(`${size}\r\n`));
                await writer.write(chunk.buffer);
                await writer.write(encode("\r\n"));

and start with this Node.js-specific WebSocket implementation https://gist.github.com/robertrypula/b813ffe23a9489bae1b677f1608676c8 and replace the Node.js-specific parts with runtime agnostic equivalent https://github.com/guest271314/direct-sockets-http-ws-server/blob/main/assets/script.js#L2-L193,

I've bundled your code - after fixing the dynamic CommonJS - to a single script. Logging stuff I see some of the code is buried in the TLS wrap code.

Resorted to currently using Transfer-Encoding: chunked where I can actually get a ReadableStream on the client side - though I have to come up with something to get input to the server. This is definitely a hack to send UDP from command line to the server which then writes to the response

await server.pipeTo(
  new WritableStream({
    async write(connection) {
      const {
        readable: client,
        writable,
        remoteAddress,
        remotePort,
      } = await connection.opened;
      console.log({ connection });
      console.log(
        { localAddress, localPort, remoteAddress, remotePort },
      );
      const writer = writable.getWriter();

      const abortable = new AbortController();
      const { signal } = abortable;

      await client.pipeTo(
        new WritableStream({
          start(controller) {
            console.log(controller);
          },
          async write(r, controller) {
            const request = decoder.decode(r);
            if (/^OPTIONS/.test(request)) {
              await writer.write(encode("HTTP/1.1 204 OK\r\n"));
              await writer.write(
                encode(
                  "Access-Control-Allow-Headers: Access-Control-Request-Private-Network\r\n",
                ),
              );
              await writer.write(
                encode("Access-Control-Allow-Origin: *\r\n"),
              );
              await writer.write(
                encode("Access-Control-Allow-Private-Network: true\r\n"),
              );
              await writer.write(
                encode(
                  "Access-Control-Allow-Headers: Access-Control-Request-Private-Network\r\n\r\n",
                ),
              );
              await writer.close();
            }
            if (/^(POST|query)/i.test(request)) {
              const [body] = request.match(
                /(?<=\r\n\r\n)[a-zA-Z\d\s\r\n-:;=]+/igm,
              );
              console.log({
                body,
              });
              await writer.write(encode("HTTP/1.1 200 OK\r\n"));
              await writer.write(
                encode("Content-Type: application/octet-stream\r\n"),
              );
              await writer.write(
                encode("Access-Control-Allow-Origin: *\r\n"),
              );
              await writer.write(
                encode("Access-Control-Allow-Private-Network: true\r\n"),
              );
              await writer.write(
                encode(
                  "Access-Control-Allow-Headers: Access-Control-Request-Private-Network\r\n",
                ),
              );
              await writer.write(encode("Cache-Control: no-cache\r\n"));
              await writer.write(encode("Connection: close\r\n"));
              await writer.write(
                encode("Transfer-Encoding: chunked\r\n\r\n"),
              );

              var localUDP = new UDPSocket({
                localPort: 10000,
                localAddress: "0.0.0.0",
              });
              console.log(localUDP);
              localUDP.closed.then(console.log).catch(console.error);
              var { readable: localReadable, writable: localWritable } =
                await localUDP.opened;
              await localReadable.pipeTo(
                new WritableStream({
                  async write({ data }) {
                    console.log(data);
                    const chunk = decoder.decode(data);
                    console.log(chunk);
                    const size = data.buffer.byteLength.toString(16);
                    await writer.write(encode(`${size}\r\n`));
                    await writer.write(data.buffer);
                    await writer.write(encode("\r\n"));
                  },
                  close() {
                    console.log("close");
                  },
                }),
              ).catch(console.log);
              /*
                await writer.write(encode("0\r\n"));
                await writer.write(encode("\r\n"));
                await writer.close();
              */
            }
          }

var abortable = new AbortController();
var {signal} = abortable;
fetch("http://0.0.0.0:44818", {
  method: "post",
  body: "test",
  headers: {
    "Access-Control-Request-Private-Network": true,
  },
  signal
})
  .then((r) => r.body.pipeThrough(new TextDecoderStream()).pipeTo(new WritableStream({
start(c) {
  console.log("start", c);
},
  write(v) {
    console.log(v);
  },
  close() {
    console.log("close");
  },
  abort(reason) {
    console.log({reason});
  }
}))
  ).catch(console.error);

deno -A --unstable-net deno-udp.js "hello, again"

I see I have to replace all of your current code that uses Buffer with TypedArray's and DataView.

An isolated article just explaining how to create the frames using runtime agostic JavaScript will be helpful. Thanks for your reply.

[HoneyryderChuck]

I'm sorry, I'm a bit confused. This repository is for a ruby library. If you require assistance for javascript, you may need to find the correct repository.

[guest271314]

I wound up here because of the article I linked to, which was very detailed re HTTP/2. I thought, maybe, just maybe, somebody might be able to explain the basic HTTP/2 header and connection process in plain text, in a programming language agnostic way. Maybe not.

[HoneyryderChuck]

For the proglang agnostic way, you have the rfc

[guest271314]

Ha, ha. I'm thinking more like how to convert the plain text version of this https://gist.github.com/rmurphey/e2308fa1eb16f07c7e4d#file-input-txt-L1-L28 to what the specification says in JavaScript.

t=    6952 [st=       0] +HTTP2_SESSION  [dt=48809520+]
                          --> host = "52.23.178.48:443"
                          --> proxy = "DIRECT"
t=    6952 [st=       0]    HTTP2_SESSION_INITIALIZED
                            --> protocol = "h2"
                            --> source_dependency = 1806765 (SOCKET)
t=    6952 [st=       0]    HTTP2_SESSION_SEND_SETTINGS
                            --> settings = ["[id:3 flags:0 value:1000]","[id:4 flags:0 value:6291456]"]
t=    6952 [st=       0]    HTTP2_STREAM_UPDATE_RECV_WINDOW
                            --> delta = 15663105
                            --> window_size = 15728640
t=    6952 [st=       0]    HTTP2_SESSION_SENT_WINDOW_UPDATE_FRAME
                            --> delta = 15663105
                            --> stream_id = 0
t=    6953 [st=       1]    HTTP2_SESSION_SEND_HEADERS
                            --> fin = true
                            --> :authority: 52.23.178.48
                                :method: GET
                                :path: /index-separate.html?push=/common/libs/combined.js
                                :scheme: https
                                accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
                                accept-encoding: gzip, deflate, sdch
                                accept-language: en-US,en;q=0.8
                                upgrade-insecure-requests: 1
                                user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36
                            --> priority = 4
                            --> stream_id = 1
                            --> unidirectional = false

It's been done, using Node.js molnarg/node-http2#104 (comment).

I'm just asking around to see if anybody is willing to spell the process out in generic terms in code. Very simple, basic, this is how to construct basic HTTP/2 headers. Then I can write out the TypedArray's in JavaScript implemented in the browser, which doesn't have Node.js API's. Thanks.

[guest271314]

For the proglang agnostic way, you have the rfc

Isn't this https://datatracker.ietf.org/doc/html/rfc9113 the specification, now?