Commit Graph

10389 Commits

Author SHA1 Message Date
Wenkai Yin
c38f969c62 Fix the consume too much CPU issue
1. Update execution status during the upgrade
2. Refine the execution sweeper

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-16 14:48:04 +08:00
Wenkai Yin
d7cc6386dd Add order by ID when listing tasks to avoid duplicated records
Add order by ID when listing tasks to avoid duplicated records

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-16 14:19:44 +08:00
Daniel Jiang
3bff4b61a6
Merge pull request #14434 from ywk253100/210312_2.2.1_retention_exec
Add upgrade sql file introduced in 2.1.4
2021-03-16 11:07:56 +08:00
danfengliu
e6c4b92913 Fix quotas test issue in upgrade pipeline
1. Change way for quotas verification in upgrade pipeline, prepare specific size of image, then it's an known value for verifcation;
2. Add notary key rotate test;
3. For issue brought by docker 20, clean containerd cache is the only effective way, so both dockerd and containerd should be cache cleard and restarted;
4. Upgrade E2E Dockerfile for importing readable file size package, and other issues;
5. Uncomment project level robot account test in nightly.
6. Get DNS from local setting, and set it into docker deamon config file;

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-03-15 10:31:32 +08:00
Wenkai Yin
b65839e90f Add upgrade sql file introduced in 2.1.4
1. Add upgrade sql file introduced in 2.1.4
2. Minor improvement for task/execution to cover corner cases

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-12 20:16:00 +08:00
Daniel Jiang
7d0cce2583
Add "*" to the claim set in the token for /v2 apis (#14343)
The "*" is used by notary server for permission checking:
84287fd8df/server/server.go (L200)
Hence, we need to add this into the JWT token such that actions like key
rotation can be executed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-03-12 15:26:19 +08:00
He Weiwei
01b3994f59 fix: search charts in the projects the user owned
Closes #14338

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-03-12 07:11:45 +00:00
Daniel Jiang
a7f2465e3a
Merge pull request #14394 from danfengliu/remove-python3.6
[Cherry-pick 2.2.0] Remove python3.6 and set dns for docker
2021-03-12 09:42:23 +08:00
danfengliu
544e1124d2 remove python3.6 and set dns for docker
Docker v20 is the default version for git action, dns should be set manually.
Python v3.8 is the default version for git action, so remove deployment for python v3.6.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-03-10 08:07:42 +00:00
danfengliu
0bc62d33a6
Merge pull request #14362 from danfengliu/cherry-pick-remove-python3.7-in-ci-installation
[cherry-pick 2.2.0] Replace python3.7 by 3.6 in CI installtion phase
2021-03-05 15:59:47 +08:00
danfengliu
168cd68b85 [cherry-pick 2.2.0] Replace python3.7 by 3.6 in CI installtion phase.
Remove python packges, and install python 3.6 to replace python 3.7.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-03-05 06:18:00 +00:00
Will Sun
340921db5e
Improve double star validator on UI (#14357)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-03-04 23:06:48 +08:00
Wang Yan
eb8b0b19bf
patch upstream fix for io reader (#14363)
Fixes #12850
This patch can fix the GC failure in the NFS v3 env, see https://github.com/distribution/distribution/pull/3309#issuecomment-783606968

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-03-04 19:19:21 +08:00
Wang Yan
155f438241
add sorting for list blobs (#14322)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-03-04 13:23:20 +08:00
danfengliu
21dc98adf7
Merge pull request #14273 from danfengliu/cherry-pick-2.2.0-fix-nightly-issues-and-upgrade-docker-and-containerd
[Cherry pick 2.2.0] fix nightly issues and upgrade docker and containerd
2021-02-24 20:47:46 +08:00
danfengliu
eee81e2670 Fix nightly issues and upgrade docker
Save base image for API tests in E2E image, it will be loaded every time
when building a local image, and all images will be cleared right after
they were pulled or pushed.

1. Fix E2E quotas issue, push the same image but with different name;
2. Add checkpoint for robot account test;
3. Upgraded docker and containerd in E2E image;
4. Package base image sample(busybox) into E2E image, so in E2E
container, all local docker images can be cleaned up, once base image is needed for
building image, it can be loaded locally;
5. Adapt OIDC service of supporting LDAP user, and add OIDC group user
test;
6. Restart docker deamon before content trust test, both in API and UI
test;
7. Add retry for keyword "Add A Tag Immutability Rule";
8. Fix tag retention test issue, missing click angle icon, and enhance
checkpoint of dry run and real run;
9. Fix schedule test issue for wrong cron string;
10. Disable quotas verification, it's not stable for script defect;

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-02-24 17:54:50 +08:00
danfengliu
a43da2aba0
Merge pull request #14227 from danfengliu/cherry-pick-2.2.0-reschedule-docker-login-policy-in-build-base-images
[Cherry-pick 2.2.0] Reschedule docker login policy in base image build process
2021-02-24 16:20:52 +08:00
danfengliu
533675b13f Upgrade docker and containerd
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-02-24 15:14:58 +08:00
danfengliu
a1bbd1a146 Reschedule docker login policy in base image build process
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-02-24 15:11:03 +08:00
Wenkai Yin(尹文开)
ec0ba1162b
Merge pull request #14285 from heww/cherry-pick-fix-issue-14251
[Cherry pick]fix: use clone query in loop of artifact.Iterator func (#14283)
2021-02-23 13:45:41 +08:00
He Weiwei
e0b3365ba8 fix: use clone query in loop of artifact.Iterator func (#14283)
Closes #14251

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-23 04:25:55 +00:00
stonezdj(Daojun Zhang)
6db1a34906
Merge pull request #14262 from heww/cherry-pick-pr-14240
[Cherry pick]chore: Bump up Trivy scanner adapter from v0.17.0 to v0.18.0 (#14240)
2021-02-19 14:53:51 +08:00
Daniel Pacak
4978a9cf4b chore: Bump up Trivy scanner adapter from v0.17.0 to v0.18.0 (#14240)
Trivy adapter version v0.18.0 ships with Trivy v0.16.0.
It also changes the way we return links to upstream
vulnerability references. Instead of returning multiple
URLs Trivy returns the primary URL to Aqua Vulnerability
Database (e.g. https://avd.aquasec.com/nvd/cve-2020-10688/)
with up to date status and remediation guide.

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
2021-02-19 04:12:56 +00:00
Will Sun
3a1110ff39
Modify cron regex for UI (#14255)
Signed-off-by: sshijun <sshijun@vmware.com>
2021-02-18 21:11:51 +08:00
Wang Yan
ef68ef30b3
fix quota dao sql injection (#14224)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-10 11:44:13 +08:00
Wenkai Yin(尹文开)
3801bbf049
Merge pull request #14222 from ywk253100/210209_sql_2.2
[cherry-pick]Fix sql issue for querying tasks
2021-02-10 10:21:08 +08:00
Wenkai Yin
247fd6df3c Fix sql issue for querying tasks
Fix sql issue for querying tasks

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-02-09 22:35:28 +08:00
Wang Yan
54b79272e0
[cherry-pick]Release 2.2.0 14195 (#14219)
* update blob list query

Deprecate blob list parameters, and use the query for instead.

Signed-off-by: wang yan <wangyan@vmware.com>

* update per review comments

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-09 20:07:39 +08:00
Daniel Jiang
66d1f76f67
Escape the service and email passed via request (#14211)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-02-09 17:07:24 +08:00
He Weiwei
0f3e112a3a
test: fix the TestRetryUntil (#14213)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-09 16:30:35 +08:00
Daniel Jiang
8c635016a8
Merge pull request #14214 from reasonerjt/use-email-from-db-2.2
[CP - v2.2]: Use the Email address from DB instead of request
2021-02-09 15:54:41 +08:00
Daniel Jiang
6f7fc44770 Use the Email address from DB instead of request
This commit updates the controller for sending reset pwd Email,
to make it use the Email from DB query result.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-02-09 14:13:06 +08:00
Wenkai Yin(尹文开)
1de9a37ddf
Merge pull request #14203 from ninjadq/metric_port_2_2_0
Fix metrics template for http mode
2021-02-08 14:02:06 +08:00
Steven Zou
a5598bfd65
Merge pull request #14202 from steven-zou/fix/chp_fix_issue_#14079
fix(jobservice):wrong depth of job logging
2021-02-08 13:44:16 +08:00
DQ
092620a0c5 Fix metrics template for http mode
the port shouldn't be hardcode

Signed-off-by: DQ <dengq@vmware.com>
2021-02-08 12:06:26 +08:00
Steven Zou
b985c41871 fix(jobservice):wrong depth of job logging
- use separate std logger for job, not shared with jobservice std logger
- merge and remove useless functions

Signed-off-by: Steven Zou <szou@vmware.com>

fix #14079
2021-02-08 11:41:28 +08:00
Wang Yan
72846163c4 rename permission of robot account (#14150)
* rename permission of robot account

fixes #14060
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-07 02:54:16 +00:00
Steven Zou
e32e0a72f4
Merge pull request #14169 from ywk253100/210204_recover
Recover the panic of the process of replication adapter and fix bug of gitlab adapter
2021-02-05 18:30:48 +08:00
He Weiwei
f870b329d2
fix: 404 if report not found when updating report (#14165)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-05 18:01:02 +08:00
Ziming Zhang
feca811894 fix(retention) migrate sql error
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-02-05 09:09:59 +00:00
Wang Yan
c2304c525d
fix gc migration issue (#14181)
For the upgrade path v1.10 - v2.1.* - v2.2.0, if user doesn't reset the GC schdule that was created in 1.10 in the v2.1,
the job parameters will keep empty in the database. The fix gives a default value for the schedule record.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-05 14:34:25 +08:00
Wenkai Yin(尹文开)
c77c9ce4d4
Merge pull request #14180 from heww/cherry-pick-fix-issue-14171
[Cherry pick]fix: wrap report vuls record creating in transaction
2021-02-05 14:04:50 +08:00
Wenkai Yin(尹文开)
92d8535464
Merge pull request #14179 from heww/cherry-pick-pr-14168
[Cherry pick]fix: remove default execution sweeper count
2021-02-05 14:04:26 +08:00
He Weiwei
6de2d4988e fix: wrap report vuls record creating in transaction
Make the creating of the ReportVulnerabilityRecord in transaction to
avoid parallel problem

Closes #14171

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-05 04:21:42 +00:00
He Weiwei
712fe37ee0 fix: remove default execution sweeper count
1. Remove the default execution sweeper count for execution vendor.
2. Set the execution sweeper count for gc, preheat, replication,
retention to 50.
3. Disable sweep for the executions of the scan job.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-05 04:18:35 +00:00
Wenkai Yin
0ac7a51d42 Recover the panic of the process of replication adapter and fix bug of gitlab adapter
Recover the panic of the process of replication adapter
Fix bug of gitlab adapter
Fixes #14153

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-02-04 18:51:43 +08:00
Will Sun
506c2d809d
Fix UI issues with label 2.2.0 (#14152)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-02-04 10:32:45 +08:00
Wenkai Yin(尹文开)
120d88a0dd
Merge pull request #14116 from reasonerjt/reduce-oidc-warning
Reduce warning logs in OIDC middleware
2021-02-01 09:55:41 +08:00
Wang Yan
1b85c67f63
fix catalog performance issue (#14120)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-31 21:31:21 +08:00
Qian Deng
a211b0c9d7
Merge pull request #14126 from ninjadq/fix_unknown_metrics
Fix: unkonw metrics issue
2021-01-29 18:59:35 +08:00