upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor synced 2024-09-20 17:45:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add CVE Allowlist expires Test Cases (#18921)

Browse Source 
Fix #18920

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
This commit is contained in:
Yang Jiao 2023-07-12 17:32:40 +08:00 committed by GitHub
parent df4dc3c00b
commit 90259f3c80
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 74 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
tests/resources/Harbor-Pages/Configuration.robot
View File

@ -277,6 +277,13 @@ Delete Top Item In System CVE Allowlist
END END
Retry Element Click ${config_security_save_button_xpath} Retry Element Click ${config_security_save_button_xpath}
Set CVE Allowlist Expires
[Arguments] ${expired}
Retry Button Click ${cve_allowlist_expires_btn}
${element}= Set Variable If ${expired} ${cve_allowlist_expires_yesterday} ${cve_allowlist_expires_tomorrow}
Retry Element Click ${element}
Retry Element Click //button[contains(.,'SAVE')]
Get Project Count Quota Text From Project Quotas List Get Project Count Quota Text From Project Quotas List
[Arguments] ${project_name} [Arguments] ${project_name}
Switch To Project Quotas Switch To Project Quotas

3
tests/resources/Harbor-Pages/Configuration_Elements.robot
View File

@ -34,6 +34,9 @@ ${configuration_system_wl_textarea} //*[@id='allowlist-textarea']
${configuration_system_wl_add_confirm_btn} //*[@id='add-to-system'] ${configuration_system_wl_add_confirm_btn} //*[@id='add-to-system']
${configuration_system_wl_delete_a_cve_id_icon} //app-security//form/section//ul/li[1]/a[2]/clr-icon ${configuration_system_wl_delete_a_cve_id_icon} //app-security//form/section//ul/li[1]/a[2]/clr-icon
${configuration_sys_repo_readonly_chb_id} //*[@id='repo_read_only_lbl'] ${configuration_sys_repo_readonly_chb_id} //*[@id='repo_read_only_lbl']
${cve_allowlist_expires_btn} //clr-date-container[.//div[@class='clr-input-group' and not(@hidden)]]//button
${cve_allowlist_expires_yesterday} //td[.//button[@class='day-btn is-today']]/preceding-sibling::td[1]
${cve_allowlist_expires_tomorrow} //td[.//button[@class='day-btn is-today']]/following-sibling::td[1]
${cfg_auth_automatic_onboarding_checkbox} //clr-checkbox-wrapper//label[contains(@for,'oidcAutoOnboard')] ${cfg_auth_automatic_onboarding_checkbox} //clr-checkbox-wrapper//label[contains(@for,'oidcAutoOnboard')]
${cfg_auth_user_name_claim_input} //*[@id='oidcUserClaim'] ${cfg_auth_user_name_claim_input} //*[@id='oidcUserClaim']

102
tests/resources/TestCaseBody.robot
View File

@ -171,37 +171,44 @@ Helm CLI Work Flow
Retry File Should Exist ./${harbor_helm_package} Retry File Should Exist ./${harbor_helm_package}
Helm Registry Logout ${ip} Helm Registry Logout ${ip}
#Important Note: All CVE IDs in CVE Allowlist cases must unique!
Body Of Verfiy System Level CVE Allowlist Body Of Verfiy System Level CVE Allowlist
[Arguments] ${image_argument} ${sha256_argument} ${most_cve_list} ${single_cve} [Arguments] ${image_argument} ${sha256_argument} ${most_cve_list} ${single_cve}
Init Chrome Driver Init Chrome Driver
${d}= Get Current Date result_format=%m%s ${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument} ${image}= Set Variable ${image_argument}
${sha256}= Set Variable ${sha256_argument} ${sha256}= Set Variable ${sha256_argument}
${signin_user}= Set Variable user025 ${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34 ${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd} Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d} Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256} Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Go Into Project project${d} Go Into Project project${d}
Set Vulnerabilty Serverity 2 Set Vulnerabilty Serverity 2
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Go Into Repo project${d} ${image} Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed Scan Repo ${sha256} Succeed
Logout Harbor Logout Harbor
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Check Listed In CVE Allowlist project${d} ${image} ${sha256} ${single_cve} is_in=No Check Listed In CVE Allowlist project${d} ${image} ${sha256} ${single_cve} is_in=No
Switch To Configuration Security Switch To Configuration Security
Retry Wait Element Visible //li[text()=' None ']
# Add Items To System CVE Allowlist CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 # Add Items To System CVE Allowlist CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528
Add Items To System CVE Allowlist ${most_cve_list} Add Items To System CVE Allowlist ${most_cve_list}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Add Items To System CVE Allowlist CVE-2021-43519 # Add Items To System CVE Allowlist CVE-2021-43519
Add Items To System CVE Allowlist ${single_cve} Add Items To System CVE Allowlist ${single_cve}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Delete Top Item In System CVE Allowlist count=9 # Set System CVE Allowlist expires to expired
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy Set CVE Allowlist Expires ${True}
Retry Wait Until Page Contains The system CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Set System CVE Allowlist expires to not expired
Set CVE Allowlist Expires ${False}
Retry Wait Until Page Does Not Contains The system CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Delete Top Item In System CVE Allowlist count=9
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Check Listed In CVE Allowlist project${d} ${image} ${sha256} ${single_cve} Check Listed In CVE Allowlist project${d} ${image} ${sha256} ${single_cve}
Close Browser Close Browser
@ -209,55 +216,70 @@ Body Of Verfiy Project Level CVE Allowlist
[Arguments] ${image_argument} ${sha256_argument} ${most_cve_list} ${single_cve} [Arguments] ${image_argument} ${sha256_argument} ${most_cve_list} ${single_cve}
[Tags] run-once [Tags] run-once
Init Chrome Driver Init Chrome Driver
${d}= Get Current Date result_format=%m%s ${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument} ${image}= Set Variable ${image_argument}
${sha256}= Set Variable ${sha256_argument} ${sha256}= Set Variable ${sha256_argument}
${signin_user}= Set Variable user025 ${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34 ${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd} Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d} Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256} Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Project project${d} Go Into Project project${d}
Set Vulnerabilty Serverity 2 Set Vulnerabilty Serverity 2
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Repo project${d} ${image} Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed Scan Repo ${sha256} Succeed
Go Into Project project${d} Go Into Project project${d}
Add Items to Project CVE Allowlist ${most_cve_list} Add Items to Project CVE Allowlist ${most_cve_list}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Add Items to Project CVE Allowlist ${single_cve} Add Items to Project CVE Allowlist ${single_cve}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
# Set System CVE Allowlist expires to expired
Set CVE Allowlist Expires ${True}
Retry Wait Until Page Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Set System CVE Allowlist expires to not expired
Set CVE Allowlist Expires ${False}
Retry Wait Until Page Does Not Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Delete Top Item In Project CVE Allowlist Delete Top Item In Project CVE Allowlist
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Close Browser Close Browser
Body Of Verfiy Project Level CVE Allowlist By Quick Way of Add System Body Of Verfiy Project Level CVE Allowlist By Quick Way of Add System
[Arguments] ${image_argument} ${sha256_argument} ${cve_list} [Arguments] ${image_argument} ${sha256_argument} ${cve_list}
[Tags] run-once [Tags] run-once
Init Chrome Driver Init Chrome Driver
${d}= Get Current Date result_format=%m%s ${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument} ${image}= Set Variable ${image_argument}
${sha256}= Set Variable ${sha256_argument} ${sha256}= Set Variable ${sha256_argument}
${signin_user}= Set Variable user025 ${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34 ${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Configuration Security Switch To Configuration Security
Add Items To System CVE Allowlist ${cve_list} Add Items To System CVE Allowlist ${cve_list}
Logout Harbor Logout Harbor
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd} Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d} Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256} Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Go Into Project project${d} Go Into Project project${d}
Set Vulnerabilty Serverity 2 Set Vulnerabilty Serverity 2
Go Into Repo project${d} ${image} Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed Scan Repo ${sha256} Succeed
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Project project${d} Go Into Project project${d}
Set Project To Project Level CVE Allowlist Set Project To Project Level CVE Allowlist
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Add System CVE Allowlist to Project CVE Allowlist By Add System Button Click Add System CVE Allowlist to Project CVE Allowlist By Add System Button Click
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
# Set System CVE Allowlist expires to expired
Set CVE Allowlist Expires ${True}
Retry Wait Until Page Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Set System CVE Allowlist expires to not expired
Set CVE Allowlist Expires ${False}
Retry Wait Until Page Does Not Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Close Browser Close Browser
Body Of Replication Of Push Images to Registry Triggered By Event Body Of Replication Of Push Images to Registry Triggered By Event

2
tests/robot-cases/Group1-Nightly/Trivy.robot
View File

@ -95,9 +95,11 @@ Test Case - Verfiy System Level CVE Allowlist
Body Of Verfiy System Level CVE Allowlist goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 CVE-2021-43519 Body Of Verfiy System Level CVE Allowlist goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 CVE-2021-43519
Test Case - Verfiy Project Level CVE Allowlist Test Case - Verfiy Project Level CVE Allowlist
[Tags] proj_cve
Body Of Verfiy Project Level CVE Allowlist goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 CVE-2021-43519 Body Of Verfiy Project Level CVE Allowlist goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 CVE-2021-43519
Test Case - Verfiy Project Level CVE Allowlist By Quick Way of Add System Test Case - Verfiy Project Level CVE Allowlist By Quick Way of Add System
[Tags] proj_cve_quick_add_sys
Body Of Verfiy Project Level CVE Allowlist By Quick Way of Add System goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 \nCVE-2021-43519 Body Of Verfiy Project Level CVE Allowlist By Quick Way of Add System goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 \nCVE-2021-43519
Test Case - Stop Scan And Stop Scan All Test Case - Stop Scan And Stop Scan All