Skip to content

URL Functions

urlParse

Simple URL Parsing

Usage

urlParse [ --help ] url [ --prefix prefix ] [ --integer-port ] [ --stringUppercase ]

Simple URL parsing. Converts a url into values which can be parsed or evaluated.

Generates output which can be evaluated and separates the URL into components.

  • url - URL
  • host - Host
  • user - User
  • password - Password
  • port - Connection port
  • name - Path with the first slash removed
  • path - Path

Does little to no validation of any characters so best used for well-formed input.

Now works on multiple URLs, output is separated by a blank line for new entries

Location: bin/build/tools/url.sh

Arguments

  • --help - Flag. Optional. Display this help.
  • url - String. Required. a Uniform Resource Locator
  • --prefix prefix - String. Optional. Prefix variable names with this string.
  • --integer-port - Flag. Optional. Force the value of port to an integer value even if the URL does not provide the port number explicitly.
  • --stringUppercase - Flag. Optional. Output variable names in upper case, not lower case (the default).

Examples

eval "$(urlParse scheme://user:password@host:port/path)"
echo $name

Return codes

  • 0 - If parsing succeeds
  • 1 - If parsing fails

urlParseItem

Get a URL component directly

Usage

urlParseItem [ component ] url ...

Extract a component from one or more URLs. Component names are the same as returned by the base urlParse function: - url - url - path - name - scheme - user - password - host - port - portDefault - error The component error changes the behavior of the function – the function succeeds and returns the error string even if the URL is invalid. This permits the retrieval of the error message without any additional formatting if needed.

Location: bin/build/tools/url.sh

Arguments

  • component - the url component to get: url, path, name, scheme, user, password, host, port, portDefault, error
  • url ... - String. URL. Required. A Uniform Resource Locator used to specify a database connection

Examples

decorate info "Connecting as $(urlParseItem user "$url")"

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

urlSchemeDefaultPort

Output the port for the given scheme

Usage

urlSchemeDefaultPort [ --help ] [ --handler handler ] scheme ...

Output the port for the given scheme

Location: bin/build/tools/url.sh

Arguments

  • --help - Flag. Optional. Display this help.
  • --handler handler - Function. Optional. Use this error handler instead of the default error handler.
  • scheme ... - String. Required. Scheme to look up the default port used for that scheme.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

urlValid

Is a URL valid?

Usage

urlValid [ --help ] url ...

Checks if a URL is syntactically valid - solely a text check.

Location: bin/build/tools/url.sh

Arguments

  • --help - Flag. Optional. Display this help.
  • url ... - String. URL. Required. A Uniform Resource Locator

Return codes

  • 0 - all URLs passed in are valid
  • 1 - at least one URL passed in is not a valid URL

urlOpen

Opens the default browser for a URL on the host operating system

Usage

urlOpen [ --help ] [ --ignore ] [ --wait ] [ --url url ]

Open a URL using the operating system. Uses the operating system's open functionality to open URLs. On Linux uses xdg-open or kde-open.

Note: Tested only on Mac OS X.

Location: bin/build/tools/url.sh

Arguments

  • --help - Flag. Optional. Display this help.
  • --ignore - Flag. Optional. Ignore any invalid URLs found.
  • --wait - Flag. When multiple URLs are passed, make a single open call with all URLs as command line arguments after all URLs are validated; otherwise each URL is opened individually with the system's open call.
  • --url url - URL. Optional. URL to open.

Reads standard input

line - URL. Optional. URL to open

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Environment

urlOpener

URL opener stream filter

Usage

urlOpener [ --exec ] [ --help ]

Open URLs which appear in a stream (but continue to output the stream)

Location: bin/build/tools/url.sh

Arguments

  • --exec - Executable. Optional. If not supplied uses urlOpen.
  • --help - Flag. Optional. Display this help.

Reads standard input

text

Writes to standard output

text

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

urlFilter

Extract URLs from arbitrary text content

Usage

urlFilter [ --show-file ] [ --file name ] [ file ]

Output URLs which appear in a stream. Takes a text file and outputs any https:// or http:// URLs found within. URLs are explicitly trimmed at quote, whitespace and escape boundaries.

Location: bin/build/tools/url.sh

Arguments

  • --show-file - Boolean. Optional. Show the file name in the output (suffix with :)
  • --file name - String. Optional. The file name to display - can be any text.
  • file - File. Optional. A file to read and output URLs found.

Reads standard input

text

Writes to standard output

line:URL

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Network access to URLs

urlFetch

Fetch URL content

Usage

urlFetch [ --help ] [ --dump headerFile ] [ --header header ] [ --wget ] [ --redirect-max maxRedirections ] [ --curl ] [ --binary binaryName ] [ --argument-format format ] [ --user userName ] [ --password password ] [ --agent userAgent ] [ --timeout timeoutSeconds ] url [ file ]

Fetch URL content

Location: bin/build/tools/url.sh

Arguments

  • --help - Flag. Optional. Display this help.
  • --dump headerFile - String. Optional. Dump the headers to the file specified, specify - to output to stdout.
  • --header header - String. Optional. Send a header in the format 'Name: Value'
  • --wget - Flag. Optional. Force use of wget. If unavailable, fail.
  • --redirect-max maxRedirections - PositiveInteger. Optional. Sets the number of allowed redirects from the original URL. Default is 9.
  • --curl - Flag. Optional. Force use of curl. If unavailable, fail.
  • --binary binaryName - Callable. Use this binary instead. If the base name of the file is not curl or wget you MUST supply --argument-format.
  • --argument-format format - String. Optional. Supply curl or wget for parameter formatting.
  • --user userName - String. Optional. If supplied, uses HTTP Simple authentication. Usually used with --password. Note: User names may not contain the character : when using curl.
  • --password password - String. Optional. If supplied along with --user, uses HTTP Simple authentication.
  • --agent userAgent - String. Optional. Specify the user agent string.
  • --timeout timeoutSeconds - PositiveInteger. Optional. A number of seconds to wait before failing. Defaults to BUILD_URL_TIMEOUT environment value.
  • url - URL. Required. URL to fetch to target file.
  • file - FileDirectory. Optional. Target file. Use - to send to stdout. Default value is -.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Environment

Requires

urlMatchesLocalFileSize

Compare a remote file size with a local file size

Usage

urlMatchesLocalFileSize [ --help ] url file

Compare a remote file size with a local file size

Location: bin/build/tools/web.sh

Arguments

  • --help - Flag. Optional. Display this help.
  • url - URL. Required. URL to check.
  • file - File. Required. File to compare.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

urlContentLength

Get the size of a remote URL

Usage

urlContentLength [ --help ] [ --handler handler ] url

Get the size of a remote URL

Location: bin/build/tools/web.sh

Arguments

  • --help - Flag. Optional. Display this help.
  • --handler handler - Function. Optional. Use this error handler instead of the default error handler.
  • url - URL. Required. URL to fetch the Content-Length.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

userAgentDefault

Default user agent string for web agents

Usage

userAgentDefault [ --help ]

A default user agent which looks more like a browser and less like a UNIX command-line tool (debatable)

Location: bin/build/tools/url.sh

Arguments

  • --help - Flag. Optional. Display this help.

Writes to standard output

String

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

🛠️ Tools · ⬅ Top