diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..4322d63 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,79 @@ +# Libcurl.js Changelog: + +## v0.5.3 (3/9/24): +- Update Wisp client and server, which improves error handling +- Expose the wisp-client-js version in the API + +## v0.5.2 (3/10/24): +- Fix a bug with error handling + +## v0.5.1 (3/10/24): +- Added support for aborting requests + +## v0.5.0 (3/9/24): +- Added support for streaming HTTP responses via a readable stream +- Improve compatibility for older browsers +- Support for all types of fetch request bodies + +## v0.4.2 (3/7/24): +- Expose a function to get error strings + +## v0.4.1 (3/7/24): +- Fix handling of duplicate HTTP headers + +## v0.4.0 (3/7/24): +- Add TLS socket support +- Add function to get the CA cert bundle +- Re-add wsproxy support +- Add custom network transport support +- Split WebSocket API into simple `libcurl.CurlWebSocket` and `libcurl.WebSocket` polyfill +- Refactor WebSocket API code + +## v0.3.9 (3/3/24): +- Fix running libcurl.js inside a web worker + +## v0.3.8 (2/28/24): +- Update Wisp client and server +- Expose Wisp client API functions + +## v0.3.7 (2/27/24): +- Pin C library versions to stable +- Load the CA certs directly from memory instead of from the Emscripten virtual filesystem + +## v0.3.6 (2/26/24): +- Fix ES6 module syntax + +## v0.3.4 (2/24/24): +- Limit max TCP connections to 50 + +## v0.3.3 (2/4/24): +- Fix a memory leak with WebSockets + +## v0.3.2 (2/4/24): +- Fix handling of 204 and 205 response codes +- Add verbose option to WebSocket API +- Fix conversion of request payloads + +## v0.3.1 (2/3/24): +- Add a copyright notice to the JS bundle + +## v0.3.0 (2/3/24): +- Add API to get the libcurl.js version and C library versions +- Add checks to ensure that the Websocket proxy URL has been set + +## v0.2.0 (2/2/24): +- Add an option to redirect the verbose curl output. +- Use separate callbacks for stdout and stderr. + +## v0.1.2 (2/1/23): +- Fix bundling the WASM into a single file +- Add unit tests + +## v0.1.1 (1/29/23): +- Don't set a default websocket proxy URL + +## v0.1.0 (1/28/23): +- Initial release on NPM +- Add Github Actions for automatic builds +- Add WebSocket support +- Add Fetch API support \ No newline at end of file diff --git a/README.md b/README.md index e9aa0d0..9c7d54f 100644 --- a/README.md +++ b/README.md @@ -2,14 +2,39 @@ This is an experimental port of [libcurl](https://curl.se/libcurl/) to WebAssembly for use in the browser. It provides an interface compatible with the [Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API), allowing you to proxy HTTPS requests from the browser with full TLS encryption. Unlike previous implementations, the proxy server cannot read the contents of your requests. +## Table of Contents: +- [Features](#features) +- [Building](#building) +- [Javascript API](#javascript-api) + * [Importing the Library](#importing-the-library) + * [Making HTTP Requests](#making-http-requests) + * [Creating WebSocket Connections](#creating-websocket-connections) + * [Using TLS Sockets](#using-tls-sockets) + * [Changing the Network Transport](#changing-the-network-transport) + * [Changing the Websocket Proxy URL](#changing-the-websocket-proxy-url) + * [Getting Libcurl's Output](#getting-libcurl-s-output) + * [Getting Error Strings](#getting-error-strings) + * [Getting Version Info](#getting-version-info) + * [Getting the CA Certificates Bundle](#getting-the-ca-certificates-bundle) +- [Proxy Server](#proxy-server) +- [Project Structure](#project-structure) +- [Copyright](#copyright) + * [Copyright Notice](#copyright-notice) + +Table of contents generated with [markdown-toc](http://ecotrust-canada.github.io/markdown-toc/). + ## Features: - Fetch compatible API - End to end encryption between the browser and the destination server - Support for up to TLS 1.3 - Support for tunneling HTTP/2 connections - Support for proxying WebSockets -- Bypass CORS restrictions +- Bypass CORS restrictions without compromising on privacy - Low latency via multiplexing and reusing open connections +- Use raw TLS sockets in the browser +- Custom network transport support +- Works inside web workers without needing special permissions or headers +- Works in all major browsers (Chromium >= 64, Firefox >= 65, Safari >= 14) ## Building: You can build this project by running the following commands: @@ -31,7 +56,7 @@ The build script will generate `client/out/libcurl.js` as well as `client/out/li ## Javascript API: ### Importing the Library: -To import the library, follow the build instructions in the previous section, and copy `client/out/libcurl.js` and `client/out/libcurl.wasm` to a directory of your choice. After the script is loaded, call `libcurl.load_wasm`, specifying the url of the `libcurl.wasm` file. +To import the library, follow the build instructions in the previous section, and copy `client/out/libcurl.js` and `client/out/libcurl.wasm` to a directory of your choice. After the script is loaded, call `libcurl.load_wasm`, specifying the url of the `libcurl.wasm` file. You do not need to call `libcurl.load_wasm` if you use the `libcurl_full.js` file, as the WASM will be bundled into the JS file. ```html @@ -51,6 +76,13 @@ document.addEventListener("libcurl_load", ()=>{ }); ``` +You may also use the, the `libcurl.onload` callback, which can be useful for running libcurl.js inside a web worker. +```js +libcurl.onload = () => { + console.log("libcurl.js ready!"); +} +``` + Once loaded, there will be a `window.libcurl` object which includes all the API functions. The `libcurl.ready` property can also be used to know if the WASM has loaded. ### Making HTTP Requests: @@ -62,12 +94,43 @@ console.log(await r.text()); Most of the standard Fetch API's features are supported, with the exception of: - CORS enforcement -- `FormData` or `URLSearchParams` as the request body - Sending credentials/cookies automatically - Caching +The response may contain multiple HTTP headers with the same name, which the `Headers` object isn't able to properly represent. If this matters to you, use `response.raw_headers`, which is an array of key value pairs, instead of `response.headers`. There is support for streaming the response body using a `ReadableStream`, as well as canceling requests using an `AbortSignal`. + +Also note that there is a hard limit of 50 active TCP connections due to emscripten limitations. + ### Creating WebSocket Connections: -To use WebSockets, create a `libcurl.WebSocket` object, which works identically to the regular [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket) object. +To use WebSockets, create a `libcurl.CurlWebSocket` object, which takes the following arguments: +- `url` - The Websocket URL. +- `protocols` - A optional list of websocket subprotocols, as an array of strings. +- `options` - An optional object with extra settings to pass to curl. + +The valid WebSocket options are: +- `headers` - HTTP request headers for the websocket handshake. +- `verbose` - A boolean flag that toggles the verbose libcurl output. This verbose output will be passed to the function defined in `libcurl.stderr`, which is `console.warn` by default. + +The following callbacks are available: +- `CurlWebSocket.onopen` - Called when the websocket is successfully connected. +- `CurlWebSocket.onmessage` - Called when a websocket message is received from the server. The data is passed to the first argument of the function, and it will be either a `Uint8Array` or a string, depending on the type of message. +- `CurlWebSocket.onclose` - Called when the websocket is cleanly closed with no error. +- `CurlWebSocket.onerror` - Called when the websocket encounters an unexpected error. The [error code](https://curl.se/libcurl/c/libcurl-errors.html) is passed to the first argument of the function. + +The `CurlWebSocket.send` function can be used to send data to the websocket. The only argument is the data that is to be sent, which must be either a string or a `Uint8Array`. + +```js +let ws = new libcurl.CurlWebSocket("wss://echo.websocket.org", [], {verbose: 1}); +ws.onopen = () => { + console.log("ws connected!"); + ws.send("hello".repeat(100)); +}; +ws.onmessage = (data) => { + console.log(data); +}; +``` + +You can also use the `libcurl.WebSocket` object, which works identically to the regular [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket) object. It uses the same arguments as the simpler `CurlWebSocket` API. ```js let ws = new libcurl.WebSocket("wss://echo.websocket.org"); ws.addEventListener("open", () => { @@ -79,7 +142,38 @@ ws.addEventListener("message", (event) => { }); ``` -### Changing the Websocket URL: +### Using TLS Sockets: +Raw TLS sockets can be created with the `libcurl.TLSSocket` class, which takes the following arguments: +- `host` - The hostname to connect to. +- `port` - The TCP port to connect to. +- `options` - An optional object with extra settings to pass to curl. + +The valid TLS socket options are: +- `verbose` - A boolean flag that toggles the verbose libcurl output. + +The callbacks work similarly to the `libcurl.CurlWebSocket` object, with the main difference being that the `onmessage` callback always returns a `Uint8Array`. + +The `TLSSocket.send` function can be used to send data to the socket. The only argument is the data that is to be sent, which must be a `Uint8Array`. + +```js +let socket = new libcurl.TLSSocket("ading.dev", 443, {verbose: 1}); +socket.onopen = () => { + console.log("socket connected!"); + let str = "GET / HTTP/1.1\r\nHost: ading.dev\r\nConnection: close\r\n\r\n"; + socket.send(new TextEncoder().encode(str)); +}; +socket.onmessage = (data) => { + console.log(new TextDecoder().decode(data)); +}; +``` + +### Changing the Network Transport: +You can change the underlying network transport by setting `libcurl.transport`. The following values are accepted: +- `"wisp"` - Use the [Wisp protocol](https://github.com/MercuryWorkshop/wisp-protocol). +- `"wsproxy"` - Use the wsproxy protocol, where a new websocket is created for each TCP connection. +- Any custom class - Use a custom network protocol. If you pass in custom code here, it must be roughly conformant with the standard `WebSocket` API. The URL that is passed into this fake websocket always looks like `"wss://example.com/ws/ading.dev:443"`, where `wss://example.com/ws/` is the proxy server URL, and `ading.dev:443` is the destination server. + +### Changing the Websocket Proxy URL: You can change the URL of the websocket proxy by using `libcurl.set_websocket`. ```js libcurl.set_websocket("ws://localhost:6001/"); @@ -87,18 +181,36 @@ libcurl.set_websocket("ws://localhost:6001/"); If the websocket proxy URL is not set and one of the other API functions is called, an error will be thrown. Note that this URL must end with a trailing slash. ### Getting Libcurl's Output: -If you want more information about a connection, you can pass the `_libcurl_verbose` argument to the `libcurl.fetch` function. +If you want more information about a connection, you can pass the `_libcurl_verbose` argument to the `libcurl.fetch` function. These are the same messages that you would see if you ran `curl -v` on the command line. ```js await libcurl.fetch("https://example.com", {_libcurl_verbose: 1}); ``` + By default this will print the output to the browser console, but you can set `libcurl.stdout` and `libcurl.stderr` to intercept these messages. This callback will be executed on every line of text that libcurl outputs. ```js libcurl.stderr = (text) => {document.body.innerHTML += text}; ``` +Libcurl.js will also output some error messages to the browser console. You can intercept these messages by setting the `libcurl.logger` callback, which takes two arguments: +- `type` - The type of message. This will be one of the following: `"log"`, `"warn"`, `"error"` +- `text` - The text that is to be logged. + +This may be useful if you are running libcurl.js inside a web worker and do not have access to the regular console API. + +### Getting Error Strings: +Libcurl.js reports errors based on the [error codes](https://curl.se/libcurl/c/libcurl-errors.html) defined by the libcurl C library. The `libcurl.get_error_string` function can be used to get an error string from an error code. + +```js +console.log(libcurl.get_error_string(56)); +//"Failure when receiving data from the peer" +``` + ### Getting Version Info: You can get version information from the `libcurl.version` object. This object will also contain the versions of all the C libraries that libcurl.js uses. `libcurl.version.lib` returns the version of libcurl.js itself. +### Getting the CA Certificates Bundle: +You can get the CA cert bundle that libcurl uses by calling `libcurl.get_cacert`. The function will return a string with the certificates in PEM format. The cert bundle comes from the [official curl website](https://curl.se/docs/caextract.html), which is extracted from the Mozilla Firefox source code. + ## Proxy Server: The proxy server consists of a standard [Wisp](https://github.com/MercuryWorkshop/wisp-protocol) server, allowing multiple TCP connections to share the same websocket. @@ -111,6 +223,17 @@ server/run.sh --static=./client For a full list of server arguments, see the [wisp-server-python documentation](https://github.com/MercuryWorkshop/wisp-server-python). +## Project Structure: +- `client` - Contains all the client-side code. + - `fragments` - Various patches for the JS that emscripten produces. The script which does the patching can be found at `client/tools/patch_js.py`. + - `javascript` - All the code for the Javascript API, and for interfacing with the compiled C code. + - `libcurl` - The C code that interfaces with the libcurl library and gets compiled by emscripten. + - `tests` - Unit tests and the scripts for running them. + - `tools` - Helper shell scripts for the build process, and for compiling the various C libraries. + - `wisp_client` - A submodule for the Wisp client library. +- `server` - Contains all the server-side code for running the websocket proxy server. + - `wisp_sever` - A submodule for the Python Wisp server. + ## Copyright: This project is licensed under the GNU AGPL v3. diff --git a/client/build.sh b/client/build.sh index 570c734..9bf1a35 100755 --- a/client/build.sh +++ b/client/build.sh @@ -78,13 +78,20 @@ rm $MODULE_FILE #add version number and copyright notice VERSION=$(cat package.json | jq -r '.version') sed -i "s/__library_version__/$VERSION/" $OUT_FILE +WISP_VERSION=$(cat $WISP_CLIENT/package.json | jq -r '.version') +sed -i "s/__wisp_version__/$WISP_VERSION/" $OUT_FILE + #add extra libraries +sed -i "/__extra_libraries__/r $JAVSCRIPT_DIR/copyright.js" $OUT_FILE sed -i "/__extra_libraries__/r $WISP_CLIENT/polyfill.js" $OUT_FILE sed -i "/__extra_libraries__/r $WISP_CLIENT/wisp.js" $OUT_FILE sed -i "/__extra_libraries__/r $JAVSCRIPT_DIR/messages.js" $OUT_FILE +sed -i "/__extra_libraries__/r $JAVSCRIPT_DIR/tls_socket.js" $OUT_FILE sed -i "/__extra_libraries__/r $JAVSCRIPT_DIR/websocket.js" $OUT_FILE -sed -i "/__extra_libraries__/r $JAVSCRIPT_DIR/copyright.js" $OUT_FILE +sed -i "/__extra_libraries__/r $JAVSCRIPT_DIR/ws_polyfill.js" $OUT_FILE +sed -i "/__extra_libraries__/r $JAVSCRIPT_DIR/util.js" $OUT_FILE +sed -i "/__extra_libraries__/r $JAVSCRIPT_DIR/logger.js" $OUT_FILE #apply patches python3 tools/patch_js.py $FRAGMENTS_DIR $OUT_FILE diff --git a/client/exported_funcs.txt b/client/exported_funcs.txt index f437e28..f925f95 100644 --- a/client/exported_funcs.txt +++ b/client/exported_funcs.txt @@ -4,11 +4,13 @@ tick_request active_requests get_version +get_cacert +get_error_str recv_from_websocket send_to_websocket close_websocket -cleanup_websocket +cleanup_handle get_result_size get_result_buffer get_result_code @@ -16,4 +18,7 @@ get_result_closed get_result_bytes_left get_result_is_text +recv_from_socket +send_to_socket + free \ No newline at end of file diff --git a/client/fragments/wisp_support.js b/client/fragments/wisp_support.js index 8ed84cd..f383585 100644 --- a/client/fragments/wisp_support.js +++ b/client/fragments/wisp_support.js @@ -1,4 +1,14 @@ /* REPLACE new WebSocketConstructor */ -new WispWebSocket \ No newline at end of file +new ((() => { + if (api.transport === "wisp") { + return WispWebSocket; + } + else if (api.transport === "wsproxy") { + return WebSocket; + } + else { //custom transports + return api.transport; + } +})()) diff --git a/client/javascript/copyright.js b/client/javascript/copyright.js index 9ad6c3e..61d1184 100644 --- a/client/javascript/copyright.js +++ b/client/javascript/copyright.js @@ -1,15 +1,10 @@ -const copyright_notice = `ading2210/libcurl.js - A port of libcurl to WASM for use in the browser. -Copyright (C) 2023 ading2210 +const copyright_notice = `libcurl.js is licensed under the GNU AGPL v3. You can find the license text and source code at the project's git repository: https://github.com/ading2210/libcurl.js -This program is free software: you can redistribute it and/or modify -it under the terms of the GNU Affero General Public License as published by -the Free Software Foundation, either version 3 of the License, or -(at your option) any later version. - -This program is distributed in the hope that it will be useful, -but WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -GNU Affero General Public License for more details. - -You should have received a copy of the GNU Affero General Public License -along with this program. If not, see .`; \ No newline at end of file +Several C libraries are used, and their licenses are listed below: +- libcurl: curl License (https://curl.se/docs/copyright.html) +- openssl: Apache License 2.0 (https://github.com/openssl/openssl/blob/master/LICENSE.txt) +- cjson: MIT License (https://github.com/DaveGamble/cJSON/blob/master/LICENSE) +- zlib: zlib License (https://www.zlib.net/zlib_license.html) +- brotli: MIT License (https://github.com/google/brotli/blob/master/LICENSE) +- nghttp2: MIT License (https://github.com/nghttp2/nghttp2/blob/master/COPYING) +`; \ No newline at end of file diff --git a/client/javascript/logger.js b/client/javascript/logger.js new file mode 100644 index 0000000..02d329c --- /dev/null +++ b/client/javascript/logger.js @@ -0,0 +1,18 @@ +function logger(type, text) { + if (type === "log") + console.log(text); + else if (type === "warn") + console.warn(text); + else if (type === "error") + console.error(text); +} + +function log_msg(text) { + logger("log", text); +} +function warn_msg(text) { + logger("warn", text); +} +function error_msg(text) { + logger("error", text); +} diff --git a/client/javascript/main.js b/client/javascript/main.js index fc8b6f8..767b903 100644 --- a/client/javascript/main.js +++ b/client/javascript/main.js @@ -30,7 +30,9 @@ var event_loop = null; var active_requests = 0; var wasm_ready = false; var version_dict = null; +var api = null; const libcurl_version = "__library_version__"; +const wisp_version = "__wisp_version__"; function check_loaded(check_websocket) { if (!wasm_ready) { @@ -41,47 +43,12 @@ function check_loaded(check_websocket) { } } -//a case insensitive dictionary for request headers -class HeadersDict { - constructor(obj) { - for (let key in obj) { - this[key] = obj[key]; - } - return new Proxy(this, this); - } - get(target, prop) { - let keys = Object.keys(this); - for (let key of keys) { - if (key.toLowerCase() === prop.toLowerCase()) { - return this[key]; - } - } - } - set(target, prop, value) { - let keys = Object.keys(this); - for (let key of keys) { - if (key.toLowerCase() === prop.toLowerCase()) { - this[key] = value; - } - } - this[prop] = value; - return true; - } -} - -function allocate_str(str) { - return allocate(intArrayFromString(str), ALLOC_NORMAL); -} - -function allocate_array(array) { - return allocate(array, ALLOC_NORMAL); -} - //low level interface with c code -function perform_request(url, params, js_data_callback, js_end_callback, body=null) { +function perform_request(url, params, js_data_callback, js_end_callback, js_headers_callback, body=null) { let params_str = JSON.stringify(params); let end_callback_ptr; let data_callback_ptr; + let headers_callback_ptr; let url_ptr = allocate_str(url); let params_ptr = allocate_str(params_str); @@ -92,30 +59,40 @@ function perform_request(url, params, js_data_callback, js_end_callback, body=nu body_length = body.length; } - let end_callback = (error, response_json_ptr) => { - let response_json = UTF8ToString(response_json_ptr); - let response_info = JSON.parse(response_json); - + function end_callback(error) { Module.removeFunction(end_callback_ptr); Module.removeFunction(data_callback_ptr); - if (body_ptr) _free(body_ptr); - _free(url_ptr); - _free(response_json_ptr); + Module.removeFunction(headers_callback_ptr); - if (error != 0) console.error("request failed with error code " + error); active_requests --; - js_end_callback(error, response_info); + js_end_callback(error); } - let data_callback = (chunk_ptr, chunk_size) => { + function data_callback(chunk_ptr, chunk_size) { let data = Module.HEAPU8.subarray(chunk_ptr, chunk_ptr + chunk_size); let chunk = new Uint8Array(data); js_data_callback(chunk); } - end_callback_ptr = Module.addFunction(end_callback, "vii"); + function headers_callback(response_json_ptr) { + let response_json = UTF8ToString(response_json_ptr); + let response_info = JSON.parse(response_json); + + if (body_ptr) _free(body_ptr); + _free(url_ptr); + _free(response_json_ptr); + + //if the response status is 0, an error occurred, + //but we don't know what it is yet + if (response_info.status !== 0) { + js_headers_callback(response_info); + } + } + + end_callback_ptr = Module.addFunction(end_callback, "vi"); + headers_callback_ptr = Module.addFunction(headers_callback, "vi"); data_callback_ptr = Module.addFunction(data_callback, "vii"); - let http_handle = _start_request(url_ptr, params_ptr, data_callback_ptr, end_callback_ptr, body_ptr, body_length); + let http_handle = _start_request(url_ptr, params_ptr, data_callback_ptr, end_callback_ptr, headers_callback_ptr, body_ptr, body_length); _free(params_ptr); active_requests ++; @@ -135,17 +112,6 @@ function perform_request(url, params, js_data_callback, js_end_callback, body=nu return http_handle; } -function merge_arrays(arrays) { - let total_len = arrays.reduce((acc, val) => acc + val.length, 0); - let new_array = new Uint8Array(total_len); - let offset = 0; - for (let array of arrays) { - new_array.set(array, offset); - offset += array.length; - } - return new_array; -} - function create_response(response_data, response_info) { response_info.ok = response_info.status >= 200 && response_info.status < 300; response_info.statusText = status_messages[response_info.status] || ""; @@ -168,71 +134,42 @@ function create_response(response_data, response_info) { writable: false, value: new Headers() }); - for (let header_name in response_info.headers) { - let header_value = response_info.headers[header_name]; + Object.defineProperty(response_obj, "raw_headers", { + writable: false, + value: response_info.headers + }); + for (let [header_name, header_value] of response_info.headers) { response_obj.headers.append(header_name, header_value); } return response_obj; } -async function parse_body(data) { - let data_array = null; - if (typeof data === "string") { - data_array = new TextEncoder().encode(data); - } - - else if (data instanceof Blob) { - let array_buffer = await data.arrayBuffer(); - data_array = new Uint8Array(array_buffer); - } - - //any typedarray - else if (data instanceof ArrayBuffer) { - //dataview objects - if (ArrayBuffer.isView(data) && data instanceof DataView) { - data_array = new Uint8Array(data.buffer); - } - //regular typed arrays - else if (ArrayBuffer.isView(data)) { - data_array = Uint8Array.from(data); - } - //regular arraybuffers - else { - data_array = new Uint8Array(data); - } - } - - else if (data instanceof ReadableStream) { - let chunks = []; - for await (let chunk of data) { - chunks.push(chunk); - } - data_array = merge_arrays(chunks); - } - - else { - throw "invalid data type to be sent"; - } - return data_array; -} - async function create_options(params) { let body = null; - if (params.body) { - body = await parse_body(params.body); - params.body = true; + let request_obj = new Request("/", params); + let array_buffer = await request_obj.arrayBuffer(); + if (array_buffer.byteLength > 0) { + body = new Uint8Array(array_buffer); } + + let headers = params.headers || {}; + if (params.headers instanceof Headers) { + for(let [key, value] of headers) { + headers[key] = value; + } + } + params.headers = new HeadersDict(headers); - if (!params.headers) params.headers = {}; - params.headers = new HeadersDict(params.headers); - - if (params.referer) { - params.headers["Referer"] = params.referer; + if (params.referrer) { + params.headers["Referer"] = params.referrer; } if (!params.headers["User-Agent"]) { params.headers["User-Agent"] = navigator.userAgent; } + if (body) { + params.headers["Content-Type"] = request_obj.headers.get("Content-Type"); + } return body; } @@ -240,22 +177,63 @@ async function create_options(params) { //wrap perform_request in a promise function perform_request_async(url, params, body) { return new Promise((resolve, reject) => { - let chunks = []; - let data_callback = (new_data) => { - chunks.push(new_data); - }; - - let finish_callback = (error, response_info) => { - if (error != 0) { - reject("libcurl.js encountered an error: " + error); - return; + let stream_controller; + let http_handle; + let response_obj; + let aborted = false; + + //handle abort signals + if (params.signal instanceof AbortSignal) { + params.signal.addEventListener("abort", () => { + if (aborted) return; + aborted = true; + _cleanup_handle(http_handle); + if (!response_obj) { + reject(new DOMException("The operation was aborted.")); + } + else { + stream_controller.error("The operation was aborted."); + } + }); + } + + let stream = new ReadableStream({ + start(controller) { + stream_controller = controller; } - let response_data = merge_arrays(chunks); - chunks = null; - let response_obj = create_response(response_data, response_info); + }); + + function data_callback(new_data) { + try { + stream_controller.enqueue(new_data); + } + catch (e) { + //the readable stream has been closed elsewhere, so cancel the request + if (e instanceof TypeError) { + _cleanup_handle(http_handle); + } + else { + throw e; + } + } + } + function headers_callback(response_info) { + response_obj = create_response(stream, response_info); resolve(response_obj); } - perform_request(url, params, data_callback, finish_callback, body); + function finish_callback(error) { + if (error != 0) { + error_msg(`Request "${url}" failed with error code ${error}: ${get_error_str(error)}`); + reject(`Request failed with error code ${error}: ${get_error_str(error)}`); + return; + } + try { + stream_controller.close(); + } //this will only fail if the stream is already errored or closed, which isn't a problem + catch {} + } + + http_handle = perform_request(url, params, data_callback, finish_callback, headers_callback, body); }); } @@ -267,12 +245,9 @@ async function libcurl_fetch(url, params={}) { function set_websocket_url(url) { websocket_url = url; - if (!Module.websocket) { - document.addEventListener("libcurl_load", () => { - set_websocket_url(url); - }); + if (Module.websocket) { + Module.websocket.url = url; } - else Module.websocket.url = url; } function get_version() { @@ -284,16 +259,24 @@ function get_version() { _free(version_ptr); version_dict = JSON.parse(version_str); version_dict.lib = libcurl_version; + version_dict.wisp = wisp_version; return version_dict; } +function get_cacert() { + return UTF8ToString(_get_cacert()); +} + function main() { wasm_ready = true; _init_curl(); set_websocket_url(websocket_url); - let load_event = new Event("libcurl_load"); - document.dispatchEvent(load_event); + if (ENVIRONMENT_IS_WEB) { + let load_event = new Event("libcurl_load"); + document.dispatchEvent(load_event); + } + api.onload(); } function load_wasm(url) { @@ -303,21 +286,35 @@ function load_wasm(url) { } Module.onRuntimeInitialized = main; -return { +api = { fetch: libcurl_fetch, set_websocket: set_websocket_url, load_wasm: load_wasm, - wisp: _wisp_connections, - WebSocket: CurlWebSocket, + WebSocket: FakeWebSocket, + CurlWebSocket: CurlWebSocket, + TLSSocket: TLSSocket, + get_cacert: get_cacert, + get_error_string: get_error_str, + + wisp_connections: _wisp_connections, + WispConnection: WispConnection, + transport: "wisp", get copyright() {return copyright_notice}, get version() {return get_version()}, get ready() {return wasm_ready}, + get websocket_url() {return websocket_url}, get stdout() {return out}, set stdout(callback) {out = callback}, get stderr() {return err}, - set stderr(callback) {err = callback} -} + set stderr(callback) {err = callback}, + get logger() {return logger}, + set logger(func) {logger = func}, + + onload() {} +}; + +return api; })() \ No newline at end of file diff --git a/client/javascript/tls_socket.js b/client/javascript/tls_socket.js new file mode 100644 index 0000000..a5249ac --- /dev/null +++ b/client/javascript/tls_socket.js @@ -0,0 +1,104 @@ +//currently broken + +class TLSSocket { + constructor(hostname, port, options={}) { + check_loaded(true); + + this.hostname = hostname; + this.port = port; + this.url = `https://${hostname}:${port}`; + this.options = options; + + this.onopen = () => {}; + this.onerror = () => {}; + this.onmessage = () => {}; + this.onclose = () => {}; + + this.connected = false; + this.event_loop = null; + + this.connect(); + } + + connect() { + let response_info; + let data_callback = () => {}; + let headers_callback = (info) => { + response_info = info; + } + let finish_callback = (error) => { + if (error === 0) { + this.connected = true; + this.event_loop = setInterval(() => { + let data = this.recv(); + if (data != null) this.onmessage(data); + }, 0); + this.onopen(); + } + else { + this.cleanup(error); + } + } + let request_options = { + _connect_only: 1, + } + if (this.options.verbose) { + request_options._libcurl_verbose = 1; + } + + this.http_handle = perform_request(this.url, request_options, data_callback, finish_callback, headers_callback, null); + } + + recv() { + let buffer_size = 64*1024; + let result_ptr = _recv_from_socket(this.http_handle, buffer_size); + let data_ptr = _get_result_buffer(result_ptr); + let result_code = _get_result_code(result_ptr); + let result_closed = _get_result_closed(result_ptr); + + if (result_code === 0 && !result_closed) { //CURLE_OK - data received + let data_size = _get_result_size(result_ptr); + let data_heap = Module.HEAPU8.subarray(data_ptr, data_ptr + data_size); + let data = new Uint8Array(data_heap); + this.onmessage(data) + } + + else if (result_code === 0 && result_closed) { + this.cleanup(); + } + + else if (result_code != 81) { + this.cleanup(result_code); + } + + _free(data_ptr); + _free(result_ptr); + } + + send(data_array) { + if (!this.connected) return; + let data_ptr = allocate_array(data_array); + let data_len = data_array.length; + _send_to_socket(this.http_handle, data_ptr, data_len); + _free(data_ptr); + } + + cleanup(error=false) { + if (this.http_handle) _cleanup_handle(this.http_handle); + else return; + + clearInterval(this.event_loop); + this.connected = false; + + if (error) { + this.onerror(error); + } + else { + this.onclose(); + } + } + + close() { + this.cleanup(); + } +} \ No newline at end of file diff --git a/client/javascript/util.js b/client/javascript/util.js new file mode 100644 index 0000000..c5557e6 --- /dev/null +++ b/client/javascript/util.js @@ -0,0 +1,74 @@ +//a case insensitive dictionary for request headers +class HeadersDict { + constructor(obj) { + for (let key in obj) { + this[key] = obj[key]; + } + return new Proxy(this, this); + } + get(target, prop) { + let keys = Object.keys(this); + for (let key of keys) { + if (key.toLowerCase() === prop.toLowerCase()) { + return this[key]; + } + } + } + set(target, prop, value) { + let keys = Object.keys(this); + for (let key of keys) { + if (key.toLowerCase() === prop.toLowerCase()) { + this[key] = value; + } + } + this[prop] = value; + return true; + } +} + +function allocate_str(str) { + return allocate(intArrayFromString(str), ALLOC_NORMAL); +} + +function allocate_array(array) { + return allocate(array, ALLOC_NORMAL); +} + +function get_error_str(error_code) { + let error_ptr = _get_error_str(error_code); + return UTF8ToString(error_ptr); +} + +function merge_arrays(arrays) { + let total_len = arrays.reduce((acc, val) => acc + val.length, 0); + let new_array = new Uint8Array(total_len); + let offset = 0; + for (let array of arrays) { + new_array.set(array, offset); + offset += array.length; + } + return new_array; +} + +//convert various data types to a uint8array (blobs excluded) +function data_to_array(data) { + //data already in correct type + if (data instanceof Uint8Array) { + return data; + } + + else if (typeof data === "string") { + return new TextEncoder().encode(data); + } + + else if (data instanceof ArrayBuffer) { + return new Uint8Array(data); + } + + //dataview objects or any other typedarray + else if (ArrayBuffer.isView(data)) { + return new Uint8Array(data.buffer); + } + + throw "invalid data type to be sent"; +} \ No newline at end of file diff --git a/client/javascript/websocket.js b/client/javascript/websocket.js index 23af368..54780de 100644 --- a/client/javascript/websocket.js +++ b/client/javascript/websocket.js @@ -1,50 +1,55 @@ -//class for custom websocket - -class CurlWebSocket extends EventTarget { - constructor(url, protocols=[], websocket_debug=false) { - super(); +class CurlWebSocket { + constructor(url, protocols=[], options={}) { check_loaded(true); if (!url.startsWith("wss://") && !url.startsWith("ws://")) { throw new SyntaxError("invalid url"); } - + this.url = url; this.protocols = protocols; - this.binaryType = "blob"; - this.recv_buffer = []; - this.websocket_debug = websocket_debug; + this.options = options; - //legacy event handlers this.onopen = () => {}; this.onerror = () => {}; this.onmessage = () => {}; this.onclose = () => {}; - this.CONNECTING = 0; - this.OPEN = 1; - this.CLOSING = 2; - this.CLOSED = 3; + this.connected = false; + this.event_loop = null; + this.recv_buffer = []; this.connect(); } connect() { - this.status = this.CONNECTING; + let response_info; let data_callback = () => {}; - let finish_callback = (error, response_info) => { - this.finish_callback(error, response_info); + let headers_callback = (info) => { + response_info = info; } - let options = {}; + let finish_callback = (error) => { + if (error === 0) { + this.connected = true; + this.event_loop = setInterval(() => { + let data = this.recv(); + if (data !== null) this.onmessage(data); + }, 0); + this.onopen(); + } + else { + this.cleanup(error); + } + } + let request_options = { + headers: this.options.headers || {} + }; if (this.protocols) { - options.headers = { - "Sec-Websocket-Protocol": this.protocols.join(", "), - }; + request_options.headers["Sec-Websocket-Protocol"] = this.protocols.join(", "); } - if (this.websocket_debug) { - options._libcurl_verbose = 1; + if (this.options.verbose) { + request_options._libcurl_verbose = 1; } - this.http_handle = perform_request(this.url, options, data_callback, finish_callback, null); - this.recv_loop(); + this.http_handle = perform_request(this.url, request_options, data_callback, finish_callback, headers_callback, null); } recv() { @@ -52,12 +57,16 @@ class CurlWebSocket extends EventTarget { let result_ptr = _recv_from_websocket(this.http_handle, buffer_size); let data_ptr = _get_result_buffer(result_ptr); let result_code = _get_result_code(result_ptr); + let result_closed = _get_result_closed(result_ptr); + let returned_data = null; - if (result_code == 0) { //CURLE_OK - data recieved + //CURLE_OK - data received + if (result_code === 0 && !result_closed) { if (_get_result_closed(result_ptr)) { - //this.pass_buffer(); - this.close_callback(); - return; + _free(data_ptr); + _free(result_ptr); + this.cleanup(); + return returned_data; } let data_size = _get_result_size(result_ptr); @@ -69,133 +78,60 @@ class CurlWebSocket extends EventTarget { let full_data = merge_arrays(this.recv_buffer); let is_text = _get_result_is_text(result_ptr) this.recv_buffer = []; - this.recv_callback(full_data, is_text); + if (is_text) { + returned_data = new TextDecoder().decode(full_data); + } + else { + returned_data = full_data; + } } } + + // websocket was cleanly closed by the server + else if (result_code === 0 && result_closed) { + this.cleanup(); + } - if (result_code == 52) { //CURLE_GOT_NOTHING - socket closed - this.close_callback(); + //code is not CURLE_AGAIN - an error must have occurred + else if (result_code !== 81) { + this.cleanup(result_code); } _free(data_ptr); _free(result_ptr); + return returned_data; } - recv_loop() { - this.event_loop = setInterval(() => { - this.recv(); - }, 1); - } - - recv_callback(data, is_text=false) { - let converted; - if (is_text) { - converted = new TextDecoder().decode(data); - } - else { - if (this.binaryType == "blob") { - converted = new Blob(data); - } - else if (this.binaryType == "arraybuffer") { - converted = data.buffer; - } - else { - throw "invalid binaryType string"; - } - } - - let msg_event = new MessageEvent("message", {data: converted}); - this.onmessage(msg_event); - this.dispatchEvent(msg_event); - } - - close_callback(error=false) { - if (this.status == this.CLOSED) return; - this.status = this.CLOSED; + cleanup(error=0) { + if (this.http_handle) _cleanup_handle(this.http_handle); + else return; clearInterval(this.event_loop); - _cleanup_websocket(); + this.connected = false; if (error) { - let error_event = new Event("error"); - this.dispatchEvent(error_event); - this.onerror(error_event); + error_msg(`Websocket "${this.url}" encountered error code ${error}: ${get_error_str(error)}`); + this.onerror(error); } else { - let close_event = new CloseEvent("close"); - this.dispatchEvent(close_event); - this.onclose(close_event); + this.onclose(); } } - finish_callback(error, response_info) { - this.status = this.OPEN; - if (error != 0) this.close_callback(true); - let open_event = new Event("open"); - this.onopen(open_event); - this.dispatchEvent(open_event); - } - send(data) { - let is_text = false; - if (this.status === this.CONNECTING) { - throw new DOMException("ws not ready yet"); - } - if (this.status === this.CLOSED) { - return; - } + let is_text = typeof data === "string"; + if (!this.connected) return; - let data_array; - if (typeof data === "string") { - data_array = new TextEncoder().encode(data); - is_text = true; + if (is_text) { + data = new TextEncoder().encode(data); } - else if (data instanceof Blob) { - data.arrayBuffer().then(array_buffer => { - data_array = new Uint8Array(array_buffer); - this.send(data_array); - }); - return; - } - //any typedarray - else if (data instanceof ArrayBuffer) { - //dataview objects - if (ArrayBuffer.isView(data) && data instanceof DataView) { - data_array = new Uint8Array(data.buffer); - } - //regular arraybuffers - else { - data_array = new Uint8Array(data); - } - } - //regular typed arrays - else if (ArrayBuffer.isView(data)) { - data_array = Uint8Array.from(data); - } - else { - throw "invalid data type to be sent"; - } - - let data_ptr = allocate_array(data_array); + let data_ptr = allocate_array(data); let data_len = data.length; _send_to_websocket(this.http_handle, data_ptr, data_len, is_text); _free(data_ptr); } close() { - _close_websocket(this.http_handle); - } - - get readyState() { - return this.status; - } - get bufferedAmount() { - return 0; - } - get protocol() { - return ""; - } - get extensions() { - return ""; + this.cleanup(); } } \ No newline at end of file diff --git a/client/javascript/ws_polyfill.js b/client/javascript/ws_polyfill.js new file mode 100644 index 0000000..2f79795 --- /dev/null +++ b/client/javascript/ws_polyfill.js @@ -0,0 +1,114 @@ +//class for websocket polyfill + +class FakeWebSocket extends EventTarget { + constructor(url, protocols=[], options={}) { + super(); + + this.url = url; + this.protocols = protocols; + this.options = options; + this.binaryType = "blob"; + + //legacy event handlers + this.onopen = () => {}; + this.onerror = () => {}; + this.onmessage = () => {}; + this.onclose = () => {}; + + this.CONNECTING = 0; + this.OPEN = 1; + this.CLOSING = 2; + this.CLOSED = 3; + this.status = this.CONNECTING; + + this.socket = null; + this.connect(); + } + + connect() { + this.socket = new CurlWebSocket(this.url, this.protocols, this.options); + + this.socket.onopen = () => { + this.status = this.OPEN; + let open_event = new Event("open"); + this.onopen(open_event); + this.dispatchEvent(open_event); + } + + this.socket.onclose = () => { + this.status = this.CLOSED; + let close_event = new CloseEvent("close"); + this.dispatchEvent(close_event); + this.onclose(close_event); + }; + + this.socket.onerror = (error) => { + this.status = this.CLOSED; + let error_event = new Event("error"); + this.dispatchEvent(error_event); + this.onerror(error_event); + } + + this.socket.onmessage = (data) => { + let converted; + if (typeof data === "string") { + converted = data; + } + else { //binary frame received as uint8array + if (this.binaryType == "blob") { + converted = new Blob(data); + } + else if (this.binaryType == "arraybuffer") { + converted = data.buffer; + } + else { + throw "invalid binaryType string"; + } + } + + let msg_event = new MessageEvent("message", {data: converted}); + this.onmessage(msg_event); + this.dispatchEvent(msg_event); + } + } + + send(data) { + if (this.status === this.CONNECTING) { + throw new DOMException("websocket not ready yet"); + } + if (this.status === this.CLOSED) { + return; + } + + if (data instanceof Blob) { + (async () => { + let array_buffer = await data.arrayBuffer(); + this.socket.send(new Uint8Array(array_buffer)); + })(); + } + else if (typeof data === "string") { + this.socket.send(data); + } + else { + this.socket.send(data_to_array(data)); + } + } + + close() { + this.status = this.CLOSING; + this.socket.close(); + } + + get readyState() { + return this.status; + } + get bufferedAmount() { + return 0; + } + get protocol() { + return this.protocols[0] || ""; + } + get extensions() { + return ""; + } +} \ No newline at end of file diff --git a/client/libcurl/main.c b/client/libcurl/main.c index 4f5578a..8b958b5 100644 --- a/client/libcurl/main.c +++ b/client/libcurl/main.c @@ -7,13 +7,14 @@ #include "curl/easy.h" #include "curl/header.h" #include "cjson/cJSON.h" -#include "cacert.h" #include "curl/multi.h" +#include "cacert.h" #include "util.h" #include "types.h" void finish_request(CURLMsg *curl_msg); +void forward_headers(struct RequestInfo *request_info); #define ERROR_REDIRECT_DISALLOWED -1 @@ -21,11 +22,15 @@ CURLM *multi_handle; int request_active = 0; struct curl_blob cacert_blob; -size_t write_function(void *data, size_t size, size_t nmemb, DataCallback data_callback) { +size_t write_function(void *data, size_t size, size_t nmemb, struct RequestInfo *request_info) { + if (!request_info->headers_received) { + forward_headers(request_info); + } + size_t real_size = size * nmemb; char* chunk = malloc(real_size); memcpy(chunk, data, real_size); - data_callback(chunk, real_size); + (*request_info->data_callback)(chunk, real_size); free(chunk); return real_size; } @@ -48,7 +53,7 @@ void tick_request() { } } -CURL* start_request(const char* url, const char* json_params, DataCallback data_callback, EndCallback end_callback, const char* body, int body_length) { +CURL* start_request(const char* url, const char* json_params, DataCallback data_callback, EndCallback end_callback, HeadersCallback headers_callback, const char* body, int body_length) { CURL *http_handle = curl_easy_init(); int abort_on_redirect = 0; int prevent_cleanup = 0; @@ -56,10 +61,6 @@ CURL* start_request(const char* url, const char* json_params, DataCallback data_ curl_easy_setopt(http_handle, CURLOPT_URL, url); curl_easy_setopt(http_handle, CURLOPT_CAINFO_BLOB , cacert_blob); - //callbacks to pass the response data back to js - curl_easy_setopt(http_handle, CURLOPT_WRITEFUNCTION, &write_function); - curl_easy_setopt(http_handle, CURLOPT_WRITEDATA, data_callback); - //some default options curl_easy_setopt(http_handle, CURLOPT_FOLLOWLOCATION, 1); curl_easy_setopt(http_handle, CURLOPT_ACCEPT_ENCODING, ""); @@ -83,6 +84,13 @@ CURL* start_request(const char* url, const char* json_params, DataCallback data_ curl_easy_setopt(http_handle, CURLOPT_VERBOSE, 1L); } + if (strcmp(key, "_connect_only") == 0) { + curl_easy_setopt(http_handle, CURLOPT_CONNECT_ONLY, 1L); + curl_easy_setopt(http_handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1); + curl_easy_setopt(http_handle, CURLOPT_SSL_ENABLE_ALPN, 0L); + prevent_cleanup = 1; + } + if (strcmp(key, "method") == 0 && cJSON_IsString(item)) { curl_easy_setopt(http_handle, CURLOPT_CUSTOMREQUEST, item->valuestring); } @@ -122,36 +130,39 @@ CURL* start_request(const char* url, const char* json_params, DataCallback data_ curl_easy_setopt(http_handle, CURLOPT_POSTFIELDSIZE, body_length); } + //create request metadata struct struct RequestInfo *request_info = malloc(sizeof(struct RequestInfo)); + request_info->http_handle = http_handle; request_info->abort_on_redirect = abort_on_redirect; request_info->curl_msg = NULL; request_info->headers_list = headers_list; - request_info->end_callback = end_callback; request_info->prevent_cleanup = prevent_cleanup; + request_info->headers_received = 0; + request_info->end_callback = end_callback; + request_info->data_callback = data_callback; + request_info->headers_callback = headers_callback; + + //callbacks to pass the response data back to js + curl_easy_setopt(http_handle, CURLOPT_WRITEFUNCTION, &write_function); + curl_easy_setopt(http_handle, CURLOPT_WRITEDATA, data_callback); + curl_easy_setopt(http_handle, CURLOPT_PRIVATE, request_info); + curl_easy_setopt(http_handle, CURLOPT_WRITEDATA, request_info); curl_multi_add_handle(multi_handle, http_handle); return http_handle; } -void finish_request(CURLMsg *curl_msg) { - //get initial request info from the http handle - struct RequestInfo *request_info; - CURL *http_handle = curl_msg->easy_handle; - curl_easy_getinfo(http_handle, CURLINFO_PRIVATE, &request_info); - - int error = (int) curl_msg->data.result; - long response_code; - curl_easy_getinfo(http_handle, CURLINFO_RESPONSE_CODE, &response_code); - - if (request_info->abort_on_redirect && response_code / 100 == 3) { - error = ERROR_REDIRECT_DISALLOWED; - } +void forward_headers(struct RequestInfo *request_info) { + request_info->headers_received = 1; + CURL *http_handle = request_info->http_handle; //create new json object with response info cJSON* response_json = cJSON_CreateObject(); + long response_code; + curl_easy_getinfo(http_handle, CURLINFO_RESPONSE_CODE, &response_code); cJSON* status_item = cJSON_CreateNumber(response_code); cJSON_AddItemToObject(response_json, "status", status_item); @@ -160,12 +171,16 @@ void finish_request(CURLMsg *curl_msg) { cJSON* url_item = cJSON_CreateString(response_url); cJSON_AddItemToObject(response_json, "url", url_item); - cJSON* headers_item = cJSON_CreateObject(); + cJSON* headers_item = cJSON_CreateArray(); struct curl_header *prev_header = NULL; struct curl_header *header = NULL; while ((header = curl_easy_nextheader(http_handle, CURLH_HEADER, -1, prev_header))) { - cJSON* header_entry = cJSON_CreateString(header->value); - cJSON_AddItemToObject(headers_item, header->name, header_entry); + cJSON* header_key_entry = cJSON_CreateString(header->name); + cJSON* header_value_entry = cJSON_CreateString(header->value); + cJSON* header_pair_item = cJSON_CreateArray(); + cJSON_AddItemToArray(header_pair_item, header_key_entry); + cJSON_AddItemToArray(header_pair_item, header_value_entry); + cJSON_AddItemToArray(headers_item, header_pair_item); prev_header = header; } cJSON_AddItemToObject(response_json, "headers", headers_item); @@ -177,10 +192,30 @@ void finish_request(CURLMsg *curl_msg) { char* response_json_str = cJSON_Print(response_json); cJSON_Delete(response_json); - + + (*request_info->headers_callback)(response_json_str); +} + +void finish_request(CURLMsg *curl_msg) { + //get initial request info from the http handle + struct RequestInfo *request_info; + CURL *http_handle = curl_msg->easy_handle; + curl_easy_getinfo(http_handle, CURLINFO_PRIVATE, &request_info); + if (!request_info->headers_received) { + forward_headers(request_info); + } + + int error = (int) curl_msg->data.result; + long response_code; + curl_easy_getinfo(http_handle, CURLINFO_RESPONSE_CODE, &response_code); + + if (request_info->abort_on_redirect && response_code / 100 == 3) { + error = ERROR_REDIRECT_DISALLOWED; + } + //clean up curl curl_slist_free_all(request_info->headers_list); - (*request_info->end_callback)(error, response_json_str); + (*request_info->end_callback)(error); if (request_info->prevent_cleanup) { return; } @@ -189,6 +224,10 @@ void finish_request(CURLMsg *curl_msg) { free(request_info); } +unsigned char* get_cacert() { + return _cacert_pem; +} + void init_curl() { curl_global_init(CURL_GLOBAL_DEFAULT); multi_handle = curl_multi_init(); diff --git a/client/libcurl/tls_socket.c b/client/libcurl/tls_socket.c new file mode 100644 index 0000000..c890769 --- /dev/null +++ b/client/libcurl/tls_socket.c @@ -0,0 +1,24 @@ +#include + +#include "curl/curl.h" +#include "curl/easy.h" +#include "types.h" + +struct WSResult* recv_from_socket(CURL* http_handle, int buffer_size) { + size_t nread; + char* buffer = malloc(buffer_size); + CURLcode res = curl_easy_recv(http_handle, buffer, buffer_size, &nread); + + struct WSResult* result = malloc(sizeof(struct WSResult)); + result->buffer_size = nread; + result->buffer = buffer; + result->res = (int) res; + result->closed = (nread == 0); + return result; +} + +int send_to_socket(CURL* http_handle, const char* data, int data_len) { + size_t sent; + CURLcode res = curl_easy_send(http_handle, data, data_len, &sent); + return (int) res; +} \ No newline at end of file diff --git a/client/libcurl/types.h b/client/libcurl/types.h index 84fb83a..b270c0d 100644 --- a/client/libcurl/types.h +++ b/client/libcurl/types.h @@ -1,12 +1,19 @@ +#include "curl/curl.h" + typedef void(*DataCallback)(char* chunk_ptr, int chunk_size); -typedef void(*EndCallback)(int error, char* response_json); +typedef void(*EndCallback)(int error); +typedef void(*HeadersCallback)(char* response_json); struct RequestInfo { + CURL* http_handle; int abort_on_redirect; int prevent_cleanup; + int headers_received; struct CURLMsg *curl_msg; struct curl_slist* headers_list; + DataCallback data_callback; EndCallback end_callback; + HeadersCallback headers_callback; }; struct WSResult { diff --git a/client/libcurl/util.c b/client/libcurl/util.c index 170d3ce..3aadde4 100644 --- a/client/libcurl/util.c +++ b/client/libcurl/util.c @@ -34,4 +34,8 @@ char* get_version() { char* version_json_str = cJSON_Print(version_json); cJSON_Delete(version_json); return version_json_str; +} + +const char* get_error_str(CURLcode error_code) { + return curl_easy_strerror(error_code); } \ No newline at end of file diff --git a/client/libcurl/websocket.c b/client/libcurl/websocket.c index 4c7e5fd..49d2ddf 100644 --- a/client/libcurl/websocket.c +++ b/client/libcurl/websocket.c @@ -37,7 +37,7 @@ void close_websocket(CURL* http_handle) { } //clean up the http handle associated with the websocket, since the main loop can't do this automatically -void cleanup_websocket(CURL* http_handle) { +void cleanup_handle(CURL* http_handle) { struct RequestInfo *request_info; curl_easy_getinfo(http_handle, CURLINFO_PRIVATE, &request_info); diff --git a/client/package.json b/client/package.json index d85ef44..0b8bdcd 100644 --- a/client/package.json +++ b/client/package.json @@ -1,6 +1,6 @@ { "name": "libcurl.js", - "version": "0.3.7", + "version": "0.5.3", "description": "An experimental port of libcurl to WebAssembly for use in the browser.", "main": "libcurl.mjs", "scripts": { diff --git a/client/publish.sh b/client/publish.sh index 2a055e0..c164ff3 100755 --- a/client/publish.sh +++ b/client/publish.sh @@ -7,6 +7,7 @@ cp package.json out cp ../README.md out cp ../LICENSE out +cp ../CHANGELOG.md out cd out npm publish \ No newline at end of file diff --git a/client/tests/run_tests.py b/client/tests/run_tests.py index 5b14bf5..19c2ad6 100644 --- a/client/tests/run_tests.py +++ b/client/tests/run_tests.py @@ -49,5 +49,8 @@ class JSTest(unittest.TestCase): def test_redirect_out(self): self.run_test("redirect_out.js") + def test_tls_socket(self): + self.run_test("test_tls_socket.js") + if __name__ == "__main__": unittest.main() \ No newline at end of file diff --git a/client/tests/scripts/test_tls_socket.js b/client/tests/scripts/test_tls_socket.js new file mode 100644 index 0000000..9335414 --- /dev/null +++ b/client/tests/scripts/test_tls_socket.js @@ -0,0 +1,27 @@ +function test() { + return new Promise((resolve, reject) => { + let socket = new libcurl.TLSSocket("cloudflare.com", 443, {verbose: 1}); + + socket.onopen = () => { + let str = "GET /cdn-cgi/trace HTTP/1.1\r\nHost: cloudflare.com\r\nConnection: close\r\n\r\n"; + socket.send(new TextEncoder().encode(str)); + }; + + socket.onmessage = (data) => { + let text = new TextDecoder().decode(data); + if (!text.includes("tls=TLSv1.3")) { + reject("cloudflare reported tls version mismatch"); + return; + } + if (!text.includes("HTTP/1.1 200 OK")) { + reject("cloudflare reported http error"); + return; + } + resolve(); + }; + + socket.onerror = (error) => { + reject("socket error occurred " + error); + } + }); +} \ No newline at end of file diff --git a/client/tools/curl.sh b/client/tools/curl.sh index 4d52f71..eeb7d9f 100755 --- a/client/tools/curl.sh +++ b/client/tools/curl.sh @@ -19,7 +19,7 @@ cd curl autoreconf -fi emconfigure ./configure --host i686-linux --disable-shared --disable-threaded-resolver --without-libpsl --disable-netrc --disable-ipv6 --disable-tftp --disable-ntlm-wb --enable-websockets --with-wolfssl=$WOLFSSL_PREFIX --with-zlib=$ZLIB_PREFIX --with-brotli=$BROTLI_PREFIX --with-nghttp2=$NGHTTP2_PREFIX -emmake make -j$CORE_COUNT CFLAGS="-Oz -pthread" LIBS="-lbrotlicommon" +emmake make -j$CORE_COUNT CFLAGS="-Oz" LIBS="-lbrotlicommon" rm -rf $PREFIX mkdir -p $PREFIX/include diff --git a/client/wisp_client b/client/wisp_client index 51ad95a..f63a01d 160000 --- a/client/wisp_client +++ b/client/wisp_client @@ -1 +1 @@ -Subproject commit 51ad95a6d912ec404c20284f0cded40c0b5c4e62 +Subproject commit f63a01d33b850f6f45a059d1f6db3c45cbe47b1e diff --git a/client/worker.html b/client/worker.html new file mode 100644 index 0000000..6cf3096 --- /dev/null +++ b/client/worker.html @@ -0,0 +1,33 @@ + + + + + + + + +

emscripten tests

+ + \ No newline at end of file diff --git a/server/wisp_server b/server/wisp_server index 3b0d432..b18670f 160000 --- a/server/wisp_server +++ b/server/wisp_server @@ -1 +1 @@ -Subproject commit 3b0d432e89cff7eaa850ba8605b180189e237f1b +Subproject commit b18670f742ec2ff8efaee6074c36dfb568f20ba2