trurl 2024 by Daniel Stenberg from curl up 2024

Daniel speaks about the trurl command line tool at curl up 2024.

1. 2024

2. Problems
Parsing URLs is difficult - especially in shell scripts
Scripts using curl often need a certain amount of URL “fiddling”
Mixing URL parsers is a known recipe for security problems
@bagder

3. Solution
A dedicated tool for URL parsing and manipulation
A ‘tr’ for URLs
Using curl’s URL parser for consistency
A curl companion tool
@bagder

4. Features
-extract any URL part
-update any part of a URL
-create a URL from components
-apply a relative URL on an absolute: “do a redirect”
-extract specific pieces of the query
-trim off specific pieces of the query
-verify that the URL is valid
-output URL components as JSON
-all of this on any number of provided URL
@bagder

5. Examples: set
Replace the hostname of a URL
$ trurl --url https://curl.se --set host=example.com
https://example.com/

Create a URL by setting components
$ trurl --set host=example.com --set scheme=ftp
ftp://example.com/


@bagder

6. Examples: get
Extract the path from a URL
$ trurl --url https://curl.se/we/are.html --get '{path}'
/we/are.html

Extract the port from a URL
$ trurl --url https://curl.se/h.html --get '{default:port}'
443
@bagder

7. Examples: query
Remove tracking tuples from query
$ trurl "https://curl.se?search=hey&utm_source=tracker"
--trim query="utm_*"
https://curl.se/?search=hey

Show a specific query key value
$ trurl "https://example.com?a=home&here=now&thisthen" -g
'{query:a}'
home
@bagder

8. Development
https://github.com/curl/trurl
Managed by the curl organization
still pre 1.0 release
140+ tests
(has helped us exercise and polish the URL parser and its API)
@bagder

9. Status
Works
Solid
Shipped by several distros
@bagder

10. Future
A one-dot-zero release
What else?
@bagder