upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor synced 2024-09-20 15:35:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add CVE Allowlist expires Test Cases (#18921)

Browse Source 
Fix #18920

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
This commit is contained in:
Yang Jiao 2023-07-12 17:32:40 +08:00 committed by GitHub
parent df4dc3c00b
commit 90259f3c80
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 74 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
tests/resources/Harbor-Pages/Configuration.robot
View File

@ -277,6 +277,13 @@ Delete Top Item In System CVE Allowlist
END
Retry Element Click ${config_security_save_button_xpath}
Set CVE Allowlist Expires
[Arguments] ${expired}
Retry Button Click ${cve_allowlist_expires_btn}
${element}= Set Variable If ${expired} ${cve_allowlist_expires_yesterday} ${cve_allowlist_expires_tomorrow}
Retry Element Click ${element}
Retry Element Click //button[contains(.,'SAVE')]
Get Project Count Quota Text From Project Quotas List
[Arguments] ${project_name}
Switch To Project Quotas

3
tests/resources/Harbor-Pages/Configuration_Elements.robot
View File

@ -34,6 +34,9 @@ ${configuration_system_wl_textarea} //*[@id='allowlist-textarea']
${configuration_system_wl_add_confirm_btn} //*[@id='add-to-system']
${configuration_system_wl_delete_a_cve_id_icon} //app-security//form/section//ul/li[1]/a[2]/clr-icon
${configuration_sys_repo_readonly_chb_id} //*[@id='repo_read_only_lbl']
${cve_allowlist_expires_btn} //clr-date-container[.//div[@class='clr-input-group' and not(@hidden)]]//button
${cve_allowlist_expires_yesterday} //td[.//button[@class='day-btn is-today']]/preceding-sibling::td[1]
${cve_allowlist_expires_tomorrow} //td[.//button[@class='day-btn is-today']]/following-sibling::td[1]
${cfg_auth_automatic_onboarding_checkbox} //clr-checkbox-wrapper//label[contains(@for,'oidcAutoOnboard')]
${cfg_auth_user_name_claim_input} //*[@id='oidcUserClaim']

102
tests/resources/TestCaseBody.robot
View File

@ -171,37 +171,44 @@ Helm CLI Work Flow
Retry File Should Exist ./${harbor_helm_package}
Helm Registry Logout ${ip}
#Important Note: All CVE IDs in CVE Allowlist cases must unique!
Body Of Verfiy System Level CVE Allowlist
[Arguments] ${image_argument} ${sha256_argument} ${most_cve_list} ${single_cve}
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument}
${sha256}= Set Variable ${sha256_argument}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Go Into Project project${d}
Set Vulnerabilty Serverity 2
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Logout Harbor
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Check Listed In CVE Allowlist project${d} ${image} ${sha256} ${single_cve} is_in=No
Switch To Configuration Security
Retry Wait Element Visible //li[text()=' None ']
# Add Items To System CVE Allowlist CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528
Add Items To System CVE Allowlist ${most_cve_list}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Add Items To System CVE Allowlist ${most_cve_list}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Add Items To System CVE Allowlist CVE-2021-43519
Add Items To System CVE Allowlist ${single_cve}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Delete Top Item In System CVE Allowlist count=9
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Add Items To System CVE Allowlist ${single_cve}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
# Set System CVE Allowlist expires to expired
Set CVE Allowlist Expires ${True}
Retry Wait Until Page Contains The system CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Set System CVE Allowlist expires to not expired
Set CVE Allowlist Expires ${False}
Retry Wait Until Page Does Not Contains The system CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Delete Top Item In System CVE Allowlist count=9
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Check Listed In CVE Allowlist project${d} ${image} ${sha256} ${single_cve}
Close Browser
@ -209,55 +216,70 @@ Body Of Verfiy Project Level CVE Allowlist
[Arguments] ${image_argument} ${sha256_argument} ${most_cve_list} ${single_cve}
[Tags] run-once
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument}
${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument}
${sha256}= Set Variable ${sha256_argument}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Project project${d}
Set Vulnerabilty Serverity 2
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Go Into Project project${d}
Add Items to Project CVE Allowlist ${most_cve_list}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Add Items to Project CVE Allowlist ${single_cve}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Add Items to Project CVE Allowlist ${most_cve_list}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Add Items to Project CVE Allowlist ${single_cve}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
# Set System CVE Allowlist expires to expired
Set CVE Allowlist Expires ${True}
Retry Wait Until Page Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Set System CVE Allowlist expires to not expired
Set CVE Allowlist Expires ${False}
Retry Wait Until Page Does Not Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Delete Top Item In Project CVE Allowlist
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Close Browser
Body Of Verfiy Project Level CVE Allowlist By Quick Way of Add System
[Arguments] ${image_argument} ${sha256_argument} ${cve_list}
[Tags] run-once
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument}
${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument}
${sha256}= Set Variable ${sha256_argument}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Configuration Security
Add Items To System CVE Allowlist ${cve_list}
Logout Harbor
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Go Into Project project${d}
Set Vulnerabilty Serverity 2
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Project project${d}
Set Project To Project Level CVE Allowlist
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Add System CVE Allowlist to Project CVE Allowlist By Add System Button Click
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
# Set System CVE Allowlist expires to expired
Set CVE Allowlist Expires ${True}
Retry Wait Until Page Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Set System CVE Allowlist expires to not expired
Set CVE Allowlist Expires ${False}
Retry Wait Until Page Does Not Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Close Browser
Body Of Replication Of Push Images to Registry Triggered By Event

2
tests/robot-cases/Group1-Nightly/Trivy.robot
View File

@ -95,9 +95,11 @@ Test Case - Verfiy System Level CVE Allowlist
Body Of Verfiy System Level CVE Allowlist goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 CVE-2021-43519
Test Case - Verfiy Project Level CVE Allowlist
[Tags] proj_cve
Body Of Verfiy Project Level CVE Allowlist goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 CVE-2021-43519
Test Case - Verfiy Project Level CVE Allowlist By Quick Way of Add System
[Tags] proj_cve_quick_add_sys
Body Of Verfiy Project Level CVE Allowlist By Quick Way of Add System goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 \nCVE-2021-43519
Test Case - Stop Scan And Stop Scan All