Skip to content

URL Functions

🛠️ Tools · ⬅ Home

Usage

urlParse [ --help ] [ url ] [ --prefix prefix ] [ --uppercase ]

Simple URL parsing. Converts a url into values which can be parsed or evaluated: - url - URL - host - Host - user - User - password - Password - port - Connection port - name - Path with the first slash removed - path - Path Does little to no validation of any characters so best used for well-formed input. Now works on multiple URLs, output is separated by a blank line for new entries

Arguments

  • --help - Flag. Optional. Display this help.
  • url - a Uniform Resource Locator
  • --prefix prefix - String. Optional. Prefix variable names with this string.
  • --uppercase - Flag. Optional. Output variable names in uppercase, not lowercase (the default).

Examples

eval "$(urlParse scheme://user:password@host:port/path)"
echo $name

Return codes

  • 0 - If parsing succeeds
  • 1 - If parsing fails

Usage

urlParseItem [ component ] url ...

Extract a component from one or more URLs

Arguments

  • component - the url component to get: url, path, name, scheme, user, password, host, port, portDefault, error
  • url ... - String. URL. Required. A Uniform Resource Locator used to specify a database connection

Examples

decorate info "Connecting as $(urlParseItem user "$url")"

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Usage

urlSchemeDefaultPort [ --help ] [ --handler handler ] scheme ...

Output the port for the given scheme

Arguments

  • --help - Flag. Optional. Display this help.
  • --handler handler - Function. Optional. Use this error handler instead of the default error handler.
  • scheme ... - String. Required. Scheme to look up the default port used for that scheme.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Usage

urlValid [ --help ] url ...

Checks if a URL is valid

Arguments

  • --help - Flag. Optional. Display this help.
  • url ... - String. URL. Required. A Uniform Resource Locator

Return codes

  • 0 - all URLs passed in are valid
  • 1 - at least one URL passed in is not a valid URL

Reads standard input

line:URL

Writes to standard output

none

Arguments

  • --help - Flag. Optional. Display this help.
  • --ignore - Flag. Optional. Ignore any invalid URLs found.
  • --wait - Flag. Optional. Display this help.
  • --url url - URL. Optional. URL to download.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Usage

urlOpener [ --exec ]

Open URLs which appear in a stream (but continue to output the stream)

Reads standard input

text

Writes to standard output

text

Arguments

  • --exec - Executable. Optional. If not supplied uses urlOpen.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Usage

urlFilter [ --show-file ] [ --file name ] [ file ]

Open URLs which appear in a stream Takes a text file and outputs any https:// or http:// URLs found within. URLs are explicitly trimmed at quote, whitespace and escape boundaries.

Reads standard input

text

Writes to standard output

line:URL

Arguments

  • --show-file - Boolean. Optional. Show the file name in the output (suffix with :)
  • --file name - String. Optional. The file name to display - can be any text.
  • file - File. Optional. A file to read and output URLs found.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Network access to URLs

Usage

urlFetch [ --help ] [ --dump headerFile ] [ --header header ] [ --wget ] [ --redirect-max maxRedirections ] [ --curl ] [ --binary binaryName ] [ --argument-format format ] [ --user userName ] [ --password password ] [ --agent userAgent ] [ --timeout timeoutSeconds ] url [ file ]

Fetch URL content

Arguments

  • --help - Flag. Optional. Display this help.
  • --dump headerFile - String. Optional. Dump the headers to the file specified, specify - to output to stdout.
  • --header header - String. Optional. Send a header in the format 'Name: Value'
  • --wget - Flag. Optional. Force use of wget. If unavailable, fail.
  • --redirect-max maxRedirections - PositiveInteger. Optional. Sets the number of allowed redirects from the original URL. Default is 9.
  • --curl - Flag. Optional. Force use of curl. If unavailable, fail.
  • --binary binaryName - Callable. Use this binary instead. If the base name of the file is not curl or wget you MUST supply --argument-format.
  • --argument-format format - String. Optional. Supply curl or wget for parameter formatting.
  • --user userName - String. Optional. If supplied, uses HTTP Simple authentication. Usually used with --password. Note: User names may not contain the character : when using curl.
  • --password password - String. Optional. If supplied along with --user, uses HTTP Simple authentication.
  • --agent userAgent - String. Optional. Specify the user agent string.
  • --timeout timeoutSeconds - PositiveInteger. Optional. A number of seconds to wait before failing. Defaults to BUILD_URL_TIMEOUT environment value.
  • url - URL. Required. URL to fetch to target file.
  • file - FileDirectory. Optional. Target file. Use - to send to stdout. Default value is -.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Environment

  • BUILD_URL_TIMEOUT - PositiveInteger. Timeout in seconds for fetching URLs in urlFetch

Requires

returnMessage executableExists decorate validate throwArgument catchArgument throwEnvironment catchEnvironment

Usage

urlMatchesLocalFileSize [ --help ] url file

Compare a remote file size with a local file size

Arguments

  • --help - Flag. Optional. Display this help.
  • url - URL. Required. URL to check.
  • file - File. Required. File to compare.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Usage

urlContentLength [ --help ] [ --handler handler ] url

Get the size of a remote URL

Arguments

  • --help - Flag. Optional. Display this help.
  • --handler handler - Function. Optional. Use this error handler instead of the default error handler.
  • url - URL. Required. URL to fetch the Content-Length.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error

Usage

userAgentDefault [ --help ]

A default user agent which looks more like a browser and less like a UNIX command-line tool (debatable)

Writes to standard output

String

Arguments

  • --help - Flag. Optional. Display this help.

Return codes

  • 0 - Success
  • 1 - Environment error
  • 2 - Argument error