1
0
Fork 0
mirror of https://github.com/gocsaf/csaf.git synced 2025-12-22 11:55:40 +01:00
Commit graph

57 commits

Author SHA1 Message Date
Sascha L. Teichmann
91ab7f6b1c
Chance supported minimal Go version back to 1.20 (#514) 2023-11-28 10:37:16 +01:00
Sascha L. Teichmann
6f8870154c Break overly long line. Fix typo in comment. 2023-11-20 21:13:24 +01:00
JanHoefelmeyer
a413852627 Downloader: Only add tlp label to path if no custom directory is configured. Refactor accordingly 2023-11-20 11:05:57 +01:00
JanHoefelmeyer
086c4ab48b
Convert a lot of command line arguments to snake case (#498)
* Convert a lot of variables to snake case

* Add snakecase for variables made out of two words that had it in no version yet (for consistency)

* Adjust example files too

---------

Co-authored-by: JanHoefelmeyer <hoefelmeyer.jan@gmail.com>
2023-11-06 14:33:05 +01:00
Sascha L. Teichmann
8f6e6ee8bb improve logging output 2023-10-17 18:52:38 +02:00
Bernhard Reiter
3923dc7044
fix: improve logging for downloader and aggregator
* use full name for printing out the used logfile for the downloader.
 * for debug or verbose, log the timeintervall that will be used
   for downloader and aggregator. (The checker has this as part
   of its output already.)
2023-10-17 11:33:03 +02:00
Sascha L. Teichmann
1cc42f0ec0
Downloader: unit test forwarder (#470)
* Simplify forward method

* Add unit test for validation status

* Add unit test for stats logging in forwarder.

* Add unit test for http client creation.

* Add unit test for replaceExt

* Add unit test for buildRequest

* Add unit test for limitedString

* Add unit test for storeFailedAdvisory

* Add unit test for storeFailedAdvisory ... fixed

* Add unit test for storeFailed

* Add unit test for forward

* comment wording
2023-09-29 09:46:51 +02:00
Sascha L. Teichmann
7a8cdb6d19
Lift distribution from v2 to v3. (#467)
* v2 -> v3

* Increase version within Makefile

---------

Co-authored-by: JanHoefelmeyer <hoefelmeyer.jan@gmail.com>
2023-09-27 15:46:57 +02:00
JanHoefelmeyer
37f770ed3d
Merge pull request #463 from csaf-poc/downloader-unit-tests
Downloader: unit tests for stats
2023-09-27 12:16:05 +02:00
Sascha L. Teichmann
7cd076d4f8
Downloader: Remove verbose flag (#464)
* Remove verbose flag from downloader.

* Do structured http logging in forwarder, too.

* Use structured logging to separate http traffic of downloader from forwarder.
2023-09-27 11:30:24 +02:00
cintek
49da14d47f
feat: log redirects (#458)
* feat: log redirects

* improved logging and renamed function
2023-09-26 10:03:09 +02:00
Sascha L. Teichmann
fca3f5bb9b Fix stats unit test. 2023-09-25 23:32:19 +02:00
Sascha L. Teichmann
b0d7dbb387 Add unit test for stats. 2023-09-25 23:23:28 +02:00
Sascha L. Teichmann
24151345f5
Drop time precision below seconds in log output. (#450) 2023-08-31 17:07:34 +02:00
Sascha L. Teichmann
018e0e55f7
Fix downloader logging docs and some comments. (#447) 2023-08-29 10:24:02 +02:00
Sascha L. Teichmann
5459f10d39
Downloader: Add structured logging, fails storing and statistics
* add  forwarding support in downloader

* Raise needed Go version to 1.21+ so slog can be used.

* Introduce validation mode flag (strict, unsafe)

* Add structured logging and place log into the download folder.

* Improve some code comment (bernhardreiter)

* Add counting stats to downloader.
2023-08-28 15:03:01 +02:00
Sascha L. Teichmann
e0475791ff
Downloader: Add forwarding to HTTP endpoint (#442)
* started with forwarding support in downloader

* Add missing files.

* Add missing files.

* Raise needed Go version

* More Go version bumping.

* Fix forwarding

* Go 1.21+ needed

* Make terminating forwarder more robust.

* Better var naming

* Remove dead code. Improve commentary.

* Prepare validation status adjustment.

* Move validations to functions to make them executable in a loop.

* Introduce validation mode flag (strict, unsafe)
2023-08-25 10:31:27 +02:00
Sascha L. Teichmann
a153906d03
Document that client-passphrase option is experimental
* Label the option experimental and limited at all places.
 * Point to the downloader documentation as single point to write more.
 * Add more hints on how the old PEM block encryption is experimental
   and why it often shall not be used.
---------

Co-authored-by: JanHoefelmeyer <hoefelmeyer.jan@gmail.com>
Co-authored-by: Bernhard Reiter <bernhard@intevation.de>
2023-08-18 11:24:39 +02:00
Sascha L. Teichmann
a7be72b740 Merge branch 'main' into downloader-client-certs 2023-08-16 11:23:14 +02:00
Bernhard Reiter
6276866cad
Merge branch 'main' into download-ignore-pattern
* Improve some text parts during the merge.
2023-08-16 08:59:42 +02:00
Sascha L. Teichmann
8c95795b96
Add option to specify download folder
* Add an option to downloader to store advisories into a given subfolder.

---------

Co-authored-by: JanHoefelmeyer <hoefelmeyer.jan@gmail.com>
Co-authored-by: Bernhard Reiter <bernhard@intevation.de>
2023-08-16 08:45:34 +02:00
Sascha L. Teichmann
d42db07bec
Fix version config and make aggreator use new command line parser. (#416)
* Fix version config and make aggreator use new command line parser.

* Replace old error check with check from the library.
2023-08-15 17:54:56 +02:00
Sascha L. Teichmann
f31ee53c27 Add client certificate support to the downloader 2023-08-02 21:16:32 +02:00
Sascha L. Teichmann
5a4e5607cb Factored out matching lists of regular expressions. 2023-08-01 17:05:49 +02:00
Sascha L. Teichmann
8aa31984df Add docs and fix output. 2023-08-01 09:54:43 +02:00
Sascha L. Teichmann
2864176111 Add ignore patterns to downloader. 2023-08-01 01:46:58 +02:00
Sascha L. Teichmann
383b0ca77b Add an option to downloader to store advisories into a given folder. 2023-07-31 17:19:38 +02:00
Sascha L. Teichmann
1d9969162f Fix version config and make aggreator use new command line parser. 2023-07-30 22:37:32 +02:00
Sascha L. Teichmann
0ad4ed9e36 Expose logging as field in AdvisoryFileProcessor to shrink constructor signature. 2023-07-26 03:31:00 +02:00
Sascha L. Teichmann
f8c3741d12 Remove stray field in config. 2023-07-26 03:29:13 +02:00
Sascha L. Teichmann
de0599ebe3 Add time interval filtering to downloader. 2023-07-26 03:22:33 +02:00
Sascha L. Teichmann
95bd705036 Factor command line parsing into an internal package for reuse. 2023-07-24 23:25:16 +02:00
Sascha L. Teichmann
81ead2776b
Use TOML as config file format in downloader (#405)
* Use TOML as config file format.

* Parse command line a second time if config file was loaded.

* Handle default values correctly.

* Use same names for config file options and command line options.
2023-07-20 11:05:20 +02:00
Sascha L. Teichmann
8630e8bac2
Add support for config files in downloader. (#404)
* Add support for config files in downloader.

* Add no-ini for the version flag, too.

* Add config file options in doc to downloader.
2023-07-19 10:49:17 +02:00
Bernhard E. Reiter
cf49c7e414
Fix go.mod and internal dependencies (#371)
* Use a "/v2" in the module path to match the git version tag which
   lead with a 2. Change all mention of the module as dependency
   internally as well.
2023-06-05 10:24:35 +02:00
Bernhard Herzog
02d476360b
Merge pull request #366 from csaf-poc/cleanup_provider_metadata_loading
Prepare infrastructure for role based reporting
2023-05-16 17:53:18 +02:00
JanHoefelmeyer
4461bd6892
Merge pull request #365 from csaf-poc/filename-id
Check that filename matches /document/tracking/id
2023-05-16 08:50:48 +02:00
Sascha L. Teichmann
2e968b197d Removed old pmd loader. 2023-05-15 08:47:18 +02:00
Sascha L. Teichmann
f32fba683d
Add concurrent downloads to downloader. (#363)
* Add concurrent downloads to downloader.

* Moved to Go 1.20

* close files channel on producer side.

* Improve error handling

* New flag to ignore signature check results. Improve docs. Do not use number of CPUs to determine number of download workers.

* Set number of default workers in downloader to two.
2023-05-02 10:10:12 +02:00
Bernhard E. Reiter
8ad805e1e5
doc: improve rate default documentation (#364)
* doc: improve rate default documentation

solve #359

* Adjust downloader doc, too.

* doc(csaf_checker): Add missing phrase, correct spelling

* docs(csaf_checker): correct format mistake

---------

Co-authored-by: Sascha L. Teichmann <sascha.teichmann@intevation.de>
Co-authored-by: tschmidtb51 <65305130+tschmidtb51@users.noreply.github.com>
2023-05-01 20:38:19 +02:00
Bernhard Herzog
a92c033a5e Check that filename matches ID in csaf_downloader 2023-04-25 18:28:07 +02:00
JanHoefelmeyer
8f87273837
Remote validator output (#347)
* The validator is now able to print the details of the remote validations.
---------

Co-authored-by: JanHoefelmeyer <hoefelmeyer.jan@gmail.com>
Co-authored-by: JanHoefelmeyer <Jan Höfelmeyer jhoefelmeyer@intevation.de>
Co-authored-by: Sascha L. Teichmann <sascha.teichmann@intevation.de>
2023-03-15 11:02:06 +01:00
Sascha L. Teichmann
ffb29f5ba4 Replace 'confirming filename' with 'conforming filename' 2023-02-03 16:21:15 +01:00
Sascha L. Teichmann
cbd9dead37 Fix small typo in reporting wromg content type 2023-02-02 00:54:39 +01:00
Sascha L. Teichmann
6430712dad Warn in checker and downloader if advisories are delivered as none 'application/json' 2023-02-02 00:54:39 +01:00
Sascha L. Teichmann
c8e0804eba Merged main into branch 2023-01-27 18:25:22 +01:00
Sascha L. Teichmann
9abc83edc1 Set the default of 'validatorpresets' to 'mandatory'. 2023-01-27 17:40:17 +01:00
Sascha L. Teichmann
51fba46893 Add extra http header support to downloader and checker. 2023-01-26 22:09:38 +01:00
Sascha L. Teichmann
bcc31c0cd6 Add remote validator support to downloader. 2023-01-25 01:12:18 +01:00
Sascha L. Teichmann
5b60e7d728 Add package comments to make revive happy. 2023-01-19 16:45:26 +01:00