1
0
Fork 0
mirror of https://github.com/gocsaf/csaf.git synced 2025-12-22 11:55:40 +01:00
Commit graph

187 commits

Author SHA1 Message Date
tschmidtb51
6b9ecead89 refactor: remove temporary solution joinUrlPath and use joinPath from Go 1.19.1 net/url 2022-09-24 18:53:03 +02:00
Sascha L. Teichmann
70b4e18b58
Treat invalid PMDs as error and stop processing if needed
*  Improve how PMD validation result is used when searching for a valid PMD.
   We now stop if no PMD can be validated against the json schema.
2022-08-26 16:53:18 +02:00
JanHoefelmeyer
3bb8ea0019
Improve checker regarding PMD location problems
* Change checking to test for Security, wellknown and DNS requirement at once and only throws error if all three fail.
* Use security.txt parser from csaf/util to extract provider url.
* Improve code comments and messages for the reports.

Co-authored-by: Jan Höfelmeyer <Jan Höfelmeyer jhoefelmeyer@intevation.de>
Co-authored-by: Sascha L. Teichmann <sascha.teichmann@intevation.de>
Co-authored-by: Bernhard Reiter <bernhard@intevation.de>
2022-08-26 13:31:56 +02:00
Sascha L. Teichmann
29d7dd1223 Make URLs absolute in integrity check 2022-08-01 14:46:57 +02:00
Sascha L. Teichmann
fad70b4dd5 Merge branch 'main' into directory-url 2022-08-01 13:20:04 +02:00
JanHoefelmeyer
9890a417b4
Merge pull request #268 from csaf-poc/no-rolie-index-changes-listings
There are no index.txt, changes.csv in ROLIE dists.
2022-08-01 13:18:36 +02:00
JanHoefelmeyer
141fbe21ca
Find missing (#269)
Solves #160 

* Implements check on whether index.txt/changes.csv and directory listings exist. Also fixes minor grammatical mistakes

* Adds missing else to prevent defaulting to missing-error handling even if another error was found

* Removes comment

* test whether changes.csv or index.txt is empty

* Fixed type mismatching, undeclared variable

* Fixes typo in variable

* Fixes another typo in variable

* Fixes formatting error

* Removed reminder comments

* Fixes formatting errors

* Added check for missing directories

* Moved empty dirlistcheck to the right position

* fixes typo

* fixes typo

* Add info if files are found

* Cleans up code

* simplified check for empty changes.csv and index.txt

Co-authored-by: Jan Höfelmeyer <Jan Höfelmeyer jhoefelmeyer@intevation.de>
Co-authored-by: Sascha L. Teichmann <sascha.teichmann@intevation.de>
2022-08-01 13:17:35 +02:00
Sascha L. Teichmann
050e225d07 Fix type assertions from directory_url expression result 2022-08-01 13:00:10 +02:00
Sascha L. Teichmann
ada8070c63 There are no index.txt, changes.csv in ROLIE dists. 2022-08-01 07:20:52 +02:00
Sascha L. Teichmann
dce3d1f4a7 load advisories via directory_urls 2022-08-01 06:46:05 +02:00
Sascha L. Teichmann
1241429d19
Load client certs early to detect misconfiguration
* Move loading of client certificate into the TLS handling code
   to an earlier position.
2022-07-26 18:00:07 +02:00
Sascha L. Teichmann
d1855a9c30
Improve checks and messages for bad entries in files
* Ignore bad URLs in index.txt, improve messages.

resolve #158
2022-07-21 17:11:46 +02:00
Sascha L. Teichmann
3a43ca5630 Merge branch 'main' into publisher-in-report 2022-07-20 11:35:17 +02:00
Fadi Abbud
6a87157184
Merge pull request #235 from csaf-poc/bad-dirs-only-once
Only report bad directories in listing check only once.
2022-07-19 12:51:29 +02:00
Sascha L. Teichmann
649b5c904b Added publisher and role to domain report. 2022-07-18 22:44:34 +02:00
Sascha L. Teichmann
efa233f2ce Merge branch 'main' into quick-ckeck 2022-07-18 20:08:02 +02:00
Sascha L. Teichmann
86a015d6bf
Correct field order in changes.csv (#228) 2022-07-18 18:37:55 +02:00
Sascha L. Teichmann
8b57851486
Moved direct loading of pmd from downloader to library. (#233)
* Moved direct loading of pmd from downloader to library,
  so aggregator and checker gain the ability.
* Disabled some checks if we were given a direct PMD URL.
2022-07-18 17:59:38 +02:00
Sascha L. Teichmann
9cba4eec30
Improve joining of url paths in some situations
* Use url.JoinPath to join URLs from a few places.
* Add util/joinpath.go from go 1.19, add the license in REUSE 3.0 compatible manner.

resolve #223

Co-authored-by: Bernhard Reiter <bernhard@intevation.de>
2022-07-18 17:41:52 +02:00
Sascha L. Teichmann
332f0b2711 Only report bad directories in listing check only once. 2022-07-18 10:27:59 +02:00
Sascha L. Teichmann
7e850f7a2f Apply date range filtering to changes.csv 2022-07-17 14:52:16 +02:00
Sascha L. Teichmann
8c53b4068b Filter ROLIE entries if we have a date range 2022-07-17 14:46:28 +02:00
Sascha L. Teichmann
b39553fc9a If using folder per year based date checks assume last second of 31 december 2022-07-17 14:35:37 +02:00
Sascha L. Teichmann
ec0548bef6 introduce quick check bases on folder year 2022-07-17 12:27:25 +02:00
JanHoefelmeyer
c00b8b37bb
Trimmed redirects (#230)
* Changes phrasing of redirects to be clearer. Now omits redirects if they are already listed as part of a larger redirect chain

* Rebuilt how the redirection string is built. Now checks for duplicate redirections after all redirections have been read

* Fixes intendation error

* Fixed redirect output.

* Fixed recording redirects.

Co-authored-by: Jan Höfelmeyer <Jan Höfelmeyer jhoefelmeyer@intevation.de>
Co-authored-by: Sascha L. Teichmann <sascha.teichmann@intevation.de>
2022-07-15 07:39:06 +02:00
JanHoefelmeyer
cbb9c7a7a1
Added path to error if ROLIE feed is broken (#206)
* Added path to error if ROLIE feed is broken

* Adds missing quotation marks which broke string

* Adjust to usual error pattern.

Co-authored-by: Jan Höfelmeyer <Jan Höfelmeyer jhoefelmeyer@intevation.de>
Co-authored-by: Sascha L. Teichmann <sascha.teichmann@intevation.de>
2022-07-11 15:46:41 +02:00
Sascha L. Teichmann
b359fd0a62
Add CSAF downloader
* Dense and refactor ROLIE code in aggregator a bit.
* Move  advisory file processor to csaf package.
* Fix minor typo on main readme
2022-06-23 14:14:44 +02:00
Sascha L. Teichmann
fa434fa039
Improve checker regarding ROLIE feed advisory URLs, hashes and signatures
* Add checking the  ROLIE feed advisory URLs, hashes and signatures.
2022-06-14 13:41:51 +02:00
Sascha L. Teichmann
589547fa94
Improve writing ROLIE feed documents
* Add signature and two time has as link rel attributes to each rolie entry
   for provider and aggregator. Thus following CSAF 2.0 csd02. 
resolve #74
2022-06-14 09:50:36 +02:00
Fadi Abbud
1344885092
Change message type of tlps fetching error (#175)
* #171
2022-06-10 12:03:08 +02:00
Sascha L. Teichmann
6ec2131f5d
Fix datetime in checker reports
* Change how the datetime is generated for the report.

Co-authored-by: Bernhard Reiter <bernhard@intevation.de>
2022-06-09 17:17:43 +02:00
Sascha L. Teichmann
922e468d99
Compare fingerprints case-insensitive 2022-06-09 16:30:20 +02:00
Sascha L. Teichmann
c09e5f66f3
Add type to checker messages
* Add a type to checker messages, so the results can be interpreted better.
   Especially the difference between warning and errors can be used.
2022-06-09 12:26:19 +02:00
Bernhard E. Reiter
a849ac0d5f
Improve https get diagnostics, add verbose option
* Implement a logging client and activate it using verbose parameter or option
   in checker and aggregator.

Co-authored-by: Sascha L. Teichmann <sascha.teichmann@intevation.de>
2022-06-02 15:07:55 +02:00
Sascha L. Teichmann
527a6f6005
Implement better search for provider-metadata.json
* Decouple loading of provider metadata from processor and moved in the base library.
* Integrate new code into checker and aggregator
* Adhere to csd02 revision of CSAF 2.0.

resolve #60
2022-05-31 18:10:18 +02:00
Sascha L. Teichmann
bc90389090 Use factored out rate throttling client in checker. 2022-05-30 23:25:21 +02:00
Fadi Abbud
a1036c3847 Add 'Rate' config option for download throttling (Checker) 2022-05-30 13:38:29 +02:00
Fadi Abbud
13423c3d4d
Add datetime and version info to report of checker
Resolve #142 

Co-authored-by: Bernhard Reiter <bernhard@intevation.de>
2022-05-24 16:59:27 +02:00
Sascha L. Teichmann
17f22855ee
Add filename conformity check
* Add util function to check a filename for confirming to csaf-v2.0-csd02.
* Add code to reject bad filenames in provider, checker, aggregator and uploader.
2022-05-20 18:57:27 +02:00
Sascha L. Teichmann
d779a3fd7e
Merge pull request #135 from csaf-poc/link-checker
Link checker
2022-05-17 17:26:35 +02:00
Sascha L. Teichmann
16556cd8bb check each advisory is listable 2022-05-17 17:20:43 +02:00
Bernhard Reiter
9eca8a924f
Improved searching for provider-metadata.json
* Changes order to try the DNS path after security.txt.
* Add diagnostic output which URL is looked for.
2022-05-17 16:08:38 +02:00
Sascha L. Teichmann
131a7155fa If there are year folders in directory listings, fetch files from the extra level. 2022-05-17 15:58:34 +02:00
Bernhard Reiter
2cfb4b8e49
Fix DNS path check
* Make it dynamic by the domain given for the check.
 * Change reporting text to be more clear about which is the dynamic
   part (in lack of direct access to the path which was checked.)
2022-05-17 15:34:39 +02:00
Bernhard Reiter
29f26e0299
Merge branch 'main' into link-checker 2022-05-17 14:45:01 +02:00
Fadi Abbud
f5b7f81aa4 Fix path for locating provider-metadata.json 2022-05-17 12:54:14 +02:00
Sascha L. Teichmann
3bbd37c441 Implemented links on directory listings checking 2022-05-17 01:22:30 +02:00
Fadi Abbud
726711c688 Implement validation for ROLIE json schema 2022-05-16 11:15:46 +02:00
Sascha L. Teichmann
8a1ebe0b7a
Add aggregator; improve itest workflow
* Factor JSON evaluation and  construction base URLs out of of checker.
* Move json path matching to util.
* Add csaf_aggregator (as additional command)
* Improve itest workflow to checkout the branch where it is running on.

resolve #105
resolve  #72

Co-authored-by: tschmidtb51 <65305130+tschmidtb51@users.noreply.github.com>
Co-authored-by: Bernhard Reiter <bernhard@intevation.de>
Co-authored-by: Fadi Abbud <fadi.abbud@intevation.de>
2022-05-10 18:12:38 +02:00
Fadi Abbud
72e6df2987 Rename some variables 2022-05-04 15:39:22 +02:00