diff --git a/CHANGELOG.md b/CHANGELOG.md index a1f99b3..bfb84cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,17 @@ # Libcurl.js Changelog: +## v0.6.19 (11/1/24): +- Improve download speeds even more by reducing memory copies +- Fix wrong mime type being reported in response blob + +## v0.6.18 (10/29/24): +- Improve download speeds by increasing libcurl's internal buffer size and listening directly for websocket messages + +## v0.6.17 (10/24/24): +- Enable a per host connection limit. This defaults to 6, which is the same as most web browsers. +- Use the latest WolfSSL again and compile with workarounds +- Upgrade to curl 8.10.1 + ## v0.6.16 (10/2/24): - Fix a bug with `Headers` objects and `Request` objects not being properly handled when passed into `libcurl.fetch` - Errors thrown are now `Error` or `TypeError` objects instead of plain strings diff --git a/README.md b/README.md index 9360016..15a9f1e 100644 --- a/README.md +++ b/README.md @@ -86,8 +86,8 @@ If you are using the single file version (`libcurl_full.js`), the `libcurl.load_ Alternatively, prebuilt versions can be found on NPM and jsDelivr. You can use the [following URLs](https://cdn.jsdelivr.net/npm/libcurl.js@latest/) to load libcurl.js from a third party CDN. ``` -https://cdn.jsdelivr.net/npm/libcurl.js@latest/libcurl.js -https://cdn.jsdelivr.net/npm/libcurl.js@latest/libcurl.wasm +https://cdn.jsdelivr.net/npm/libcurl.js@0.6.7/libcurl.js +https://cdn.jsdelivr.net/npm/libcurl.js@0.6.7/libcurl.wasm ``` To know when libcurl.js has finished loading, you can use the `libcurl_load` DOM event. The `libcurl_abort` event will trigger if the Emscripten runtime gets aborted due to a critical error. The `libcurl.events` object contains an `EventTarget` where these events will also be emitted. @@ -131,9 +131,9 @@ Most of the standard Fetch API's features are supported, with the exception of: Sending cookies is supported, but they will not be automatically sent unless you create a new HTTP session, which is covered in the next section. -The response may contain multiple HTTP headers with the same name, which the `Headers` object isn't able to properly represent. If this matters to you, use `response.raw_headers`, which is an array of key value pairs, instead of `response.headers`. There is support for streaming the response body using a `ReadableStream`, as well as canceling requests using an `AbortSignal`. All requests made using this method share the same connection pool, which has a limit of 50 active TCP connections. +The response may contain multiple HTTP headers with the same name, which the `Headers` object isn't able to properly represent. If this matters to you, use `response.raw_headers`, which is an array of key value pairs, instead of `response.headers`. There is support for streaming the response body using a `ReadableStream`, as well as canceling requests using an `AbortSignal`. All requests made using this method share the same connection pool, which has a limit of 50 active TCP connections (configurable if you use a separate HTTP session). -The `proxy` option may be used to specify the URL of a `socks5h`, `socks4a`, or `http` proxy server. For example `proxy: "socks5h://127.0.0.0:1080"` will set the proxy server for just the current request. +The `proxy` option may be used to specify the URL of a `socks5h`, `socks4a`, or `http` proxy server. For example `proxy: "socks5h://127.0.0.0:1080"` will set the proxy server for just the current request. This proxy option is separate from the global websocket proxy. ### Creating New HTTP Sessions: To create new sessions for HTTP requests, use the `libcurl.HTTPSession` class. The constructor for this class takes the following arguments: @@ -146,7 +146,7 @@ The valid HTTP session settings are: Each HTTP session has the following methods available: - `fetch` - Identical to the `libcurl.fetch` function but only creates connections in this session. -- `set_connections` - Set the connection limits. This takes two arguments, the first being the limit for the connection cache, and the second being the max number of active connections. +- `set_connections` - Set the connection limits. This takes three arguments: the first [is the hard limit of active connections](https://curl.se/libcurl/c/CURLMOPT_MAX_TOTAL_CONNECTIONS.html) (default 60), the second is [limit for the connection cache](https://curl.se/libcurl/c/CURLMOPT_MAXCONNECTS.html) (default 50), and the third is the [max connections per host](https://curl.se/libcurl/c/CURLMOPT_MAX_HOST_CONNECTIONS.html) (default 6). - `export_cookies` - Export any cookies which were recorded in the session. This will return an empty string if cookies are disabled or no cookies have been set yet. - `close` - Close all connections and clean up the session. You must call this after you are done using the session, otherwise it will leak memory. @@ -246,6 +246,8 @@ libcurl.set_websocket("ws://localhost:6001/"); ``` If the websocket proxy URL is not set and one of the other API functions is called, an error will be thrown. Note that this URL must end with a trailing slash. +Changing the websocket proxy URL will not close any existing connections. + ### Getting Libcurl's Output: If you want more information about a connection, you can pass the `_libcurl_verbose` argument to the `libcurl.fetch` function. These are the same messages that you would see if you ran `curl -v` on the command line. ```js @@ -278,7 +280,7 @@ You can get version information from the `libcurl.version` object. This object w You can get the CA cert bundle that libcurl uses by calling `libcurl.get_cacert`. The function will return a string with the certificates in PEM format. The cert bundle comes from the [official curl website](https://curl.se/docs/caextract.html), which is extracted from the Mozilla Firefox source code. ### Using the Wisp Client -The `libcurl.wisp` object exposes all of the APIs from [wisp-client-js](https://github.com/MercuryWorkshop/wisp-client-js). This API is not guarenteed to be reliable. +The `libcurl.wisp` object exposes all of the APIs from [wisp-client-js](https://github.com/MercuryWorkshop/wisp-client-js). This API is not guaranteed to be stable. ## Proxy Server: The proxy server consists of a standard [Wisp](https://github.com/MercuryWorkshop/wisp-protocol) server, allowing multiple TCP connections to share the same websocket. @@ -309,7 +311,7 @@ This project is licensed under the GNU AGPL v3. ### Copyright Notice: ``` ading2210/libcurl.js - A port of libcurl to WASM for use in the browser. -Copyright (C) 2023 ading2210 +Copyright (C) 2024 ading2210 This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by diff --git a/client/javascript/http.js b/client/javascript/http.js index fa08dfe..dda47b6 100644 --- a/client/javascript/http.js +++ b/client/javascript/http.js @@ -4,7 +4,7 @@ class HTTPSession extends CurlSession { this.options = options; this.base_url = undefined; - this.set_connections(50, 40); + this.set_connections(50, 40, 6); this.import_cookies(); } @@ -181,6 +181,17 @@ class HTTPSession extends CurlSession { for (let [header_name, header_value] of response_info.headers) { response_obj.headers.append(header_name, header_value); } + + //hack to fix invalid blob type + let response_proto = Object.getPrototypeOf(response_obj); + response_obj.blob = async () => { + let blob = await response_proto.blob.call(response_obj); + let mime_type = blob.type.split(";")[0].trim(); + Object.defineProperty(blob, "type", { + value: mime_type + }); + return blob; + } return response_obj; } diff --git a/client/javascript/main.js b/client/javascript/main.js index 87c71ed..2222224 100644 --- a/client/javascript/main.js +++ b/client/javascript/main.js @@ -1,6 +1,6 @@ /* ading2210/libcurl.js - A port of libcurl to WASM for the browser. -Copyright (C) 2023 ading2210 +Copyright (C) 2024 ading2210 This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by diff --git a/client/javascript/session.js b/client/javascript/session.js index 568f79a..29eb06e 100644 --- a/client/javascript/session.js +++ b/client/javascript/session.js @@ -28,9 +28,9 @@ class CurlSession { } } - set_connections(connections_limit, cache_limit) { + set_connections(connections_limit, cache_limit, host_conn_limit=0) { this.assert_ready(); - _session_set_options(this.session_ptr, connections_limit, cache_limit); + _session_set_options(this.session_ptr, connections_limit, cache_limit, host_conn_limit); } end_callback(request_id, error) { @@ -41,8 +41,7 @@ class CurlSession { data_callback(request_id, chunk_ptr, chunk_size) { let data = Module.HEAPU8.subarray(chunk_ptr, chunk_ptr + chunk_size); - let chunk = new Uint8Array(data); - this.request_callbacks[request_id].data(chunk); + this.request_callbacks[request_id].data(data); } headers_callback(request_id, chunk_ptr, chunk_size) { diff --git a/client/libcurl/request.c b/client/libcurl/request.c index 8472ba7..501e583 100644 --- a/client/libcurl/request.c +++ b/client/libcurl/request.c @@ -40,6 +40,7 @@ CURL* create_request(const char* url, int request_id, DataCallback data_callback curl_easy_setopt(http_handle, CURLOPT_PRIVATE, request_info); curl_easy_setopt(http_handle, CURLOPT_URL, url); + curl_easy_setopt(http_handle, CURLOPT_BUFFERSIZE, 512*1024); //callbacks to pass the response data back to js curl_easy_setopt(http_handle, CURLOPT_WRITEFUNCTION, &write_function); diff --git a/client/libcurl/session.c b/client/libcurl/session.c index fc28b09..f20948f 100644 --- a/client/libcurl/session.c +++ b/client/libcurl/session.c @@ -27,9 +27,10 @@ void session_perform(struct SessionInfo *session) { } } -void session_set_options(struct SessionInfo *session, int connections_limit, int cache_limit) { +void session_set_options(struct SessionInfo *session, int connections_limit, int cache_limit, int host_conn_limit) { curl_multi_setopt(session->multi_handle, CURLMOPT_MAX_TOTAL_CONNECTIONS, connections_limit); curl_multi_setopt(session->multi_handle, CURLMOPT_MAXCONNECTS, cache_limit); + curl_multi_setopt(session->multi_handle, CURLMOPT_MAX_HOST_CONNECTIONS, host_conn_limit); } void session_add_request(struct SessionInfo *session, CURL* http_handle) { diff --git a/client/package.json b/client/package.json index 8889af3..db1a88a 100644 --- a/client/package.json +++ b/client/package.json @@ -1,6 +1,6 @@ { "name": "libcurl.js-reclaimtls", - "version": "0.6.16-reclaimtls", + "version": "0.6.19-reclaimtls", "description": "A port of libcurl to WebAssembly, for proxying HTTPS requests from the browser with full TLS encryption", "main": "libcurl.mjs", "exports": { diff --git a/server/wisp_server b/server/wisp_server index 78f872c..f931adf 160000 --- a/server/wisp_server +++ b/server/wisp_server @@ -1 +1 @@ -Subproject commit 78f872c8e5b180e26ec09cc126e841e12ac75c36 +Subproject commit f931adf9fea476fefffb1f0c3789cafda58d54ba