Merge branch 'main' into sha-handling

docs/csaf_downloader.md

@@ -1,4 +1,5 @@
 ## csaf_downloader
+
 A tool to download CSAF documents from CSAF providers.
 
 ### Usage
@@ -21,6 +22,7 @@ Application Options:
   -f, --folder=FOLDER                            Download into a given subFOLDER
   -i, --ignore_pattern=PATTERN                   Do not download files if their URLs match any of the given PATTERNs
   -H, --header=                                  One or more extra HTTP header fields
+      --enumerate_pmd_only                       If this flag is set to true, the downloader will only enumerate valid provider metadata files, but not download documents
       --validator=URL                            URL to validate documents remotely
       --validator_cache=FILE                     FILE to cache remote validations
       --validator_preset=PRESETS                 One or more PRESETS to validate remotely (default: [mandatory])
@@ -29,8 +31,8 @@ Application Options:
       --forward_header=                          One or more extra HTTP header fields used by forwarding
       --forward_queue=LENGTH                     Maximal queue LENGTH before forwarder (default: 5)
       --forward_insecure                         Do not check TLS certificates from forward endpoint
-      --logfile=FILE                             FILE to log downloading to (default: downloader.log)
-      --loglevel=LEVEL[debug|info|warn|error]    LEVEL of logging details (default: info)
+      --log_file=FILE                            FILE to log downloading to (default: downloader.log)
+      --log_level=LEVEL[debug|info|warn|error]   LEVEL of logging details (default: info)
   -c, --config=TOML-FILE                         Path to config TOML file
       --preferred_hash=HASH[sha256|sha512]       HASH to prefer
 
@@ -40,6 +42,8 @@ Help Options:
 
 Will download all CSAF documents for the given _domains_, by trying each as a CSAF provider.
 
+If no user agent is specified with `--header=user-agent:custom-agent/1.0` then the default agent in the form of `csaf_distribution/VERSION` is sent.
+
 If a _domain_ starts with `https://` it is instead considered a direct URL to the `provider-metadata.json` and downloading procedes from there.
 
 Increasing the number of workers opens more connections to the web servers
@@ -48,6 +52,7 @@ However, since this also increases the load on the servers, their administrators
 have taken countermeasures to limit this.
 
 If no config file is explictly given the follwing places are searched for a config file:
+
 ```
 ~/.config/csaf/downloader.toml
 ~/.csaf_downloader.toml
@@ -57,6 +62,7 @@ csaf_downloader.toml
 with `~` expanding to `$HOME` on unixoid systems and `%HOMEPATH` on Windows systems.
 
 Supported options in config files:
+
 ```
 # directory         # not set by default
 insecure            = false
@@ -91,6 +97,7 @@ option.
 E.g. `-i='.*white.*' -i='*.red.*'` will ignore files which URLs contain
 the sub strings **white** or **red**.
 In the config file this has to be noted as:
+
 ```
 ignorepattern = [".*white.*", ".*red.*"]
 ```
@@ -107,16 +114,18 @@ into a given intervall. There are three possible notations:
    and 'y' for years are recognized. In these cases only integer
    values are accepted without any fractions.
    Some examples:
+
    - `"3h"` means downloading the advisories that have changed in the last three hours.
-   - `"30m"`  .. changed within the last thirty minutes.
+   - `"30m"` .. changed within the last thirty minutes.
    - `"3M2m"` .. changed within the last three months and two minutes.
-   - `"2y"`   .. changed within the last two years.
+   - `"2y"` .. changed within the last two years.
 
 2. Absolute. If the given string is an RFC 3339 date timestamp
    the time interval between this date and now is used.
    E.g. `"2006-01-02"` means that all files between 2006 January 2nd and now going to being
    downloaded.
    Accepted patterns are:
+
    - `"2006-01-02T15:04:05Z"`
    - `"2006-01-02T15:04:05+07:00"`
    - `"2006-01-02T15:04:05-07:00"`
@@ -135,6 +144,7 @@ into a given intervall. There are three possible notations:
 All interval boundaries are inclusive.
 
 #### Forwarding
+
 The downloader is able to forward downloaded advisories and their checksums,
 OpenPGP signatures and validation results to an HTTP endpoint.  
 The details of the implemented API are described [here](https://github.com/mfd2007/csaf_upload_interface).  
@@ -150,7 +160,7 @@ key protection mechanism based on RFC 1423, see
 Thus it considered experimental and most likely to be removed
 in a future release. Please only use this option, if you fully understand
 the security implications!
-Note that for fully automated processes, it usually not make sense
+Note that for fully automated processes, it usually does not make sense
 to protect the client certificate's private key with a passphrase.
 Because the passphrase has to be accessible to the process anyway to run
 unattented. In this situation the processing environment should be secured