Bernhard Reiter
6276866cad
Merge branch 'main' into download-ignore-pattern
...
* Improve some text parts during the merge.
2023-08-16 08:59:42 +02:00
Sascha L. Teichmann
8c95795b96
Add option to specify download folder
...
* Add an option to downloader to store advisories into a given subfolder.
---------
Co-authored-by: JanHoefelmeyer <hoefelmeyer.jan@gmail.com>
Co-authored-by: Bernhard Reiter <bernhard@intevation.de>
2023-08-16 08:45:34 +02:00
Sascha L. Teichmann
d42db07bec
Fix version config and make aggreator use new command line parser. ( #416 )
...
* Fix version config and make aggreator use new command line parser.
* Replace old error check with check from the library.
2023-08-15 17:54:56 +02:00
Sascha L. Teichmann
f31ee53c27
Add client certificate support to the downloader
2023-08-02 21:16:32 +02:00
Sascha L. Teichmann
5a4e5607cb
Factored out matching lists of regular expressions.
2023-08-01 17:05:49 +02:00
Sascha L. Teichmann
8aa31984df
Add docs and fix output.
2023-08-01 09:54:43 +02:00
Sascha L. Teichmann
2864176111
Add ignore patterns to downloader.
2023-08-01 01:46:58 +02:00
Sascha L. Teichmann
383b0ca77b
Add an option to downloader to store advisories into a given folder.
2023-07-31 17:19:38 +02:00
Sascha L. Teichmann
1d9969162f
Fix version config and make aggreator use new command line parser.
2023-07-30 22:37:32 +02:00
Sascha L. Teichmann
0ad4ed9e36
Expose logging as field in AdvisoryFileProcessor to shrink constructor signature.
2023-07-26 03:31:00 +02:00
Sascha L. Teichmann
f8c3741d12
Remove stray field in config.
2023-07-26 03:29:13 +02:00
Sascha L. Teichmann
de0599ebe3
Add time interval filtering to downloader.
2023-07-26 03:22:33 +02:00
Sascha L. Teichmann
95bd705036
Factor command line parsing into an internal package for reuse.
2023-07-24 23:25:16 +02:00
Sascha L. Teichmann
81ead2776b
Use TOML as config file format in downloader ( #405 )
...
* Use TOML as config file format.
* Parse command line a second time if config file was loaded.
* Handle default values correctly.
* Use same names for config file options and command line options.
2023-07-20 11:05:20 +02:00
Sascha L. Teichmann
8630e8bac2
Add support for config files in downloader. ( #404 )
...
* Add support for config files in downloader.
* Add no-ini for the version flag, too.
* Add config file options in doc to downloader.
2023-07-19 10:49:17 +02:00
Bernhard E. Reiter
cf49c7e414
Fix go.mod and internal dependencies ( #371 )
...
* Use a "/v2" in the module path to match the git version tag which
lead with a 2. Change all mention of the module as dependency
internally as well.
2023-06-05 10:24:35 +02:00
Bernhard Herzog
02d476360b
Merge pull request #366 from csaf-poc/cleanup_provider_metadata_loading
...
Prepare infrastructure for role based reporting
2023-05-16 17:53:18 +02:00
JanHoefelmeyer
4461bd6892
Merge pull request #365 from csaf-poc/filename-id
...
Check that filename matches /document/tracking/id
2023-05-16 08:50:48 +02:00
Sascha L. Teichmann
2e968b197d
Removed old pmd loader.
2023-05-15 08:47:18 +02:00
Sascha L. Teichmann
f32fba683d
Add concurrent downloads to downloader. ( #363 )
...
* Add concurrent downloads to downloader.
* Moved to Go 1.20
* close files channel on producer side.
* Improve error handling
* New flag to ignore signature check results. Improve docs. Do not use number of CPUs to determine number of download workers.
* Set number of default workers in downloader to two.
2023-05-02 10:10:12 +02:00
Bernhard E. Reiter
8ad805e1e5
doc: improve rate default documentation ( #364 )
...
* doc: improve rate default documentation
solve #359
* Adjust downloader doc, too.
* doc(csaf_checker): Add missing phrase, correct spelling
* docs(csaf_checker): correct format mistake
---------
Co-authored-by: Sascha L. Teichmann <sascha.teichmann@intevation.de>
Co-authored-by: tschmidtb51 <65305130+tschmidtb51@users.noreply.github.com>
2023-05-01 20:38:19 +02:00
Bernhard Herzog
a92c033a5e
Check that filename matches ID in csaf_downloader
2023-04-25 18:28:07 +02:00
JanHoefelmeyer
8f87273837
Remote validator output ( #347 )
...
* The validator is now able to print the details of the remote validations.
---------
Co-authored-by: JanHoefelmeyer <hoefelmeyer.jan@gmail.com>
Co-authored-by: JanHoefelmeyer <Jan Höfelmeyer jhoefelmeyer@intevation.de>
Co-authored-by: Sascha L. Teichmann <sascha.teichmann@intevation.de>
2023-03-15 11:02:06 +01:00
Sascha L. Teichmann
ffb29f5ba4
Replace 'confirming filename' with 'conforming filename'
2023-02-03 16:21:15 +01:00
Sascha L. Teichmann
cbd9dead37
Fix small typo in reporting wromg content type
2023-02-02 00:54:39 +01:00
Sascha L. Teichmann
6430712dad
Warn in checker and downloader if advisories are delivered as none 'application/json'
2023-02-02 00:54:39 +01:00
Sascha L. Teichmann
c8e0804eba
Merged main into branch
2023-01-27 18:25:22 +01:00
Sascha L. Teichmann
9abc83edc1
Set the default of 'validatorpresets' to 'mandatory'.
2023-01-27 17:40:17 +01:00
Sascha L. Teichmann
51fba46893
Add extra http header support to downloader and checker.
2023-01-26 22:09:38 +01:00
Sascha L. Teichmann
bcc31c0cd6
Add remote validator support to downloader.
2023-01-25 01:12:18 +01:00
Sascha L. Teichmann
5b60e7d728
Add package comments to make revive happy.
2023-01-19 16:45:26 +01:00
Sascha L. Teichmann
c4b70d20cd
Demand Go 1.19 in go.mod. Replaced interface{} with any
2023-01-19 16:45:26 +01:00
Sascha L. Teichmann
70b4e18b58
Treat invalid PMDs as error and stop processing if needed
...
* Improve how PMD validation result is used when searching for a valid PMD.
We now stop if no PMD can be validated against the json schema.
2022-08-26 16:53:18 +02:00
Sascha L. Teichmann
d1855a9c30
Improve checks and messages for bad entries in files
...
* Ignore bad URLs in index.txt, improve messages.
resolve #158
2022-07-21 17:11:46 +02:00
Sascha L. Teichmann
a1d609c7d8
when downloader is run with verbose flag the advisory validation issues are logged in detail.
2022-07-20 12:49:50 +02:00
Sascha L. Teichmann
8b57851486
Moved direct loading of pmd from downloader to library. ( #233 )
...
* Moved direct loading of pmd from downloader to library,
so aggregator and checker gain the ability.
* Disabled some checks if we were given a direct PMD URL.
2022-07-18 17:59:38 +02:00
Sascha L. Teichmann
f8ce08a26e
Download advisories to label/year folders
2022-06-23 15:04:49 +02:00
Sascha L. Teichmann
b359fd0a62
Add CSAF downloader
...
* Dense and refactor ROLIE code in aggregator a bit.
* Move advisory file processor to csaf package.
* Fix minor typo on main readme
2022-06-23 14:14:44 +02:00