Commit Graph

306 Commits

Author SHA1 Message Date
danfengliu
d4b7888098
Merge pull request #13882 from danfengliu/fix-nightly-keyword-issues
Add tests and fix nightly issues
2021-01-19 17:28:52 +08:00
Danfeng Liu (c)
ddf9d74135 Fix nightly keyword issues
1. Add retry for Get Statics Text keywords make this operation robust;
2. Replace image to one with slash in name for CNAB test, to cover more
condition;
3. Add replication rule deletion verfication in upgrade test;
4. Add non-admin user in pull operation for proxy cache test;
5. Add verification for quota display for upgrade test;
6. Add test for large size of image replication;
7. Add test checkpoint for system robot account py-test script;

Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2021-01-19 15:12:15 +08:00
Qian Deng
d06ce6efd5
Merge pull request #13922 from ninjadq/add_e2e_test_for_metrics
Add e2e test for harbor metrics
2021-01-18 11:04:11 +08:00
Ziming Zhang
5bff3e94a0 fix(immutability) swagger test case
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-11 03:21:12 +00:00
Ziming Zhang
be58c1d3ff fix(retention) swagger test case
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-11 03:21:12 +00:00
DQ
27a9f71e26 Add e2e test for harbor metrics
Signed-off-by: DQ <dengq@vmware.com>
2021-01-09 04:12:32 +08:00
Ziming Zhang
39fb500318 feat(retention) refactor to use go swagger api
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-08 07:09:28 +00:00
danfengliu
f57dc13081 Add py-test for system robot account
In this py-test, add a new system robot account, and verify all project privileges.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-08 12:01:53 +08:00
danfengliu
511bd86930
Merge pull request #13849 from danfengliu/add-gui-test-for-system-robot-account
Ad  nightly GUI test for system robot account
2021-01-06 16:59:05 +08:00
Danfeng Liu (c)
ae05e3fdb0 Add Nightly GUI test for system robot account
1. Add GUI test for system robot account feature;
2. Fix issue of failing to catch docker pull image exception in API test.

Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2021-01-06 15:47:04 +08:00
danfengliu
076ebb5ee8 Clear local images in nightly test
Remove local image right after pushing operation, then pull operation can be triggered by docker 20.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-04 10:25:02 +08:00
danfengliu
3659abc62a Migrate gitlab replication test to nightly
Add 2 gitlab pull/push replication test scripts in replication pipeline.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-11 18:04:46 +08:00
danfengliu
ad7581d9b3 Migrate gitlab replication test to nightly
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-09 12:00:38 +08:00
danfengliu
56a35437b4
Change image source in e2e pytest (#13640)
Change source of most of test image samples from docker-hub to local building ones, so it will cost less docker-hub pull requests.
And some of cases like push cnab, they have to use docker-hub, but image samples in cnab test will cost 17 quotas, in this PR, we
replace those samples, now cnab case will cost 6 quotas.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-04 02:28:29 -08:00
DQ
590212b485 Remove clair related code
- clair code in harbor core
- clair code in frontend
- clair code in robotcase

Signed-off-by: DQ <dengq@vmware.com>
2020-11-27 14:01:04 +08:00
danfengliu
2bf89e9356 Add docker-hub login before docker-hub pull
Due to docker-hub pull request rate limitation, we will use registed account to pull image
from docker-hub, therefore add docker-hub login in API tests.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-22 21:11:05 +08:00
danfengliu
a70753d382 Add p2p python test script
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-11 14:55:23 +08:00
danfengliu
1b127b16c5 Add API test scripts for tag immutability and scan signed image
1. Add 7 API python test scripts for tag immutability;
2. Swagger file has object struct defination issue on imuutability policy rule, because
   this policy is in shared with retention rule, therefore, they should be unified to
   the same  type of rule;
3. Fixed a timeout issue of action keyword, waiting for a long period;
4. Add API test scripts for scan signed image.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-03 10:10:06 +00:00
danfengliu
691168b8cf
Merge pull request #13359 from danfengliu/optimize-upgrade-pipeline-of-pull-push-image
Optimize upgrade pipeline script of push image
2020-10-29 14:31:30 +08:00
danfengliu
ab3e24a2e3 Add proxy cache python test scipts and upgrade containerd
Containerd in e2e image was using native one in ubuntu, it should be updated to the latest release.
And add proxy cache py-tests including pull image/manifest list by docker and ctr CLI.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-10-28 02:13:40 +00:00
danfengliu
cff4f529cd Optimize upgrade pipeline script of push image
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-10-26 15:16:18 +08:00
danfengliu
4bc7444602 Add UI test for LDAP admin group dn setting
1. Verify LDAP group admin in UI;
2. Update LDAP group admin need to verify the AdminRoleInAuth, get current user will return this field as true.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-10-14 11:05:09 +08:00
danfengliu
e4894017cd Fix typo in upgrade prepare scritp
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-09-28 10:36:22 +08:00
danfengliu
48510fde08
Merge branch 'master' into add-2.1-upgrade-pipeline-in-nightly 2020-09-22 13:48:22 +08:00
danfengliu
ef53edf553 Recovery nightly setup
1. Recovery nightly setup;
2. Add retry for replication rule selection.
3. Add retry for CNAB test.
4. Rerun keyword of push CNAB

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-09-22 02:41:50 +00:00
danfengliu
e2989e3b93 Add 2.0 & 2.1 upgrade pipeline in nightly
1. Add 2.0 upgrade pipeline in nightly;
2. Add 2.1 upgrade pipeline in nightly.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-09-22 02:14:40 +00:00
danfengliu
86fb6fdc65 Add checkpoint for LDAP group py-test
LDAP group has different role, user in group has the same role, as groups with different roles were added in project
member list, user should act like in different roles. for admin and dev role, there should be checkpoints to verify priviledges of each own.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-09-14 21:15:09 +08:00
danfengliu
931bde0df8 Fix issue of missing swagger package for python API test
1. Python API test can't import packages from swagger maker, the reason is that python install packages to site-packages instead of dist-packages,
   so the current solution is to copy pakages from site back to dist, I will try to find a way of using ENV variable to fix this;

2. Remove a python library no longer be used.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-09-02 15:18:23 +08:00
danfengliu
00180b2a6f Add P2P Preheat UI Test
1. Fix ldap script issue, test case name should be in lowercase;
2. Add manifest list checkout point for proxy cache issue;
3. Add P2P Preheat policy CRUD Test.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-08-28 11:34:32 +08:00
stonezdj(Daojun Zhang)
b2ed195df5
Merge pull request #12785 from danfengliu/add-distribution-ui-test
Add distribution UI test in nightly
2020-08-21 14:19:05 +08:00
danfengliu
37e280c49f Add proxy-cache and distribution UI test in nightly
1. Add proxy-cache UI test in nightly;
2. Add distribution UI test in nightly.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-08-21 02:08:06 +00:00
He Weiwei
f659523f50 fix(scan): fix the permission checking for artifact scanning
Closes #12778

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-08-17 09:55:38 +00:00
danfengliu
e1eac2b4d0
Merge pull request #12688 from danfengliu/fix-upgrade-test-wrong-keyword-issue
Fix wrong keyword issue in upgrade test
2020-08-14 10:11:37 +08:00
danfengliu
9a2188641d Upgrade python and robot for drone build
1.Separate Dockerfile to drone and nightly e2e, one Dockerfile should cover
 both scenarios, it will be resolved later;
2.Fix docker api err type issue, API error has no message attibute;
3.Upgrade docker in nightly e2e images.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-08-12 10:26:12 +00:00
danfengliu
ab7767f539 Fix wrong keyword issue in upgrade test
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-08-10 14:47:38 +00:00
danfengliu
4968b6619b Recovery drone build script
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-08-10 14:23:44 +00:00
Flávio Ramalho
3d3fabbd20
Make notary-remove-image-signature more flexible (#12536)
Instead of explicitly defining the expect path to /usr/local/bin/expect,
use the env command, so the expect executable is searched for and launched
from wherever it is first found.

Signed-off-by: Flávio Ramalho <framalho@suse.com>
2020-07-31 15:49:45 +08:00
danfengliu
8328e95b63 Add GCS&S3 push-based replicatiton tests
1. Add GCS push-based replicatiton tests;
   There is an issue against it, so it's commented out;
2. Add S3 push-based replicatiton tests;
3. Add filter registry keyword;

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-07-29 12:44:49 +00:00
danfengliu
8a691c5df1 Add OIDC user onboard test in nightly
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-07-27 13:56:18 +08:00
danfengliu
e20cb51d6b Fix issues in replication tests
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-07-20 09:42:55 +08:00
danfengliu
aa43afb601 Add push-based replication test in nightly
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-07-16 19:50:42 +08:00
danfengliu
0cb378d8fa Verify error message when deleting signed image
1. Verify error message when deleting signed image;
2. Add test case - Replication From Dockerhub;

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-07-06 10:56:31 +08:00
AllForNothing
fff6f7529a Replace all whitelist with allowlist
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-06-24 16:17:17 +08:00
danfengliu
a8f159728b Add API E2E pytest of GC with untag flag enabled
1. Fix issue of keyword Go Into Repo, the verification logic could be more strict;
2. Add API E2E pytest of GC with untag flag enabled;
3. Add sleep in test_user_view_logs.py for delete log ocurred;
4. Test Case - Tag CRUD is not stable. Although add button was clicked, but the tag was'nt added successfully.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-23 14:48:04 +08:00
danfengliu
67ccc98c58 Add helm and helm3 CLI py-test
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-16 16:27:31 +08:00
danfengliu
4b5080d44e Add singularity py-test
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-10 11:09:41 +08:00
danfengliu
e020b90bf0 add oras cli py-test
1. Add oras cli py-test;
2. Add env for notary url, allow to input different notary port instead of solid 4443;
3. Add retry for keyword Cannot Pull Image and make it longer during retry.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-08 11:10:07 +08:00
danfengliu
3f2850ffdf Scan result behavior changed after trivy upgrade
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-04 16:25:21 +08:00
danfengliu
ceaa0a57b3 Add replication schedule test
1. Add replication schedule test;
2. Add some sleep after project creation before push image to avoid push image v2 error.
   It fixed issue# 12094: Scan image vmware/photon:1.0 return unsupported occasionally.
3. Add some sleep in test_user_view_logs.py, can't get operation logs sometimes.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-02 15:51:10 +08:00
danfengliu
5d8a308158 Add retry for keyword go into index
1. Add retry for keyword go into index;
2. Add retry for keyword go into project.
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-05-27 08:55:58 +00:00